System and method for multicasting multimedia content

ABSTRACT

A multicast network system utilizes a high speed link, such as a satellite link, to multicast multimedia information from the Internet to a plurality of receivers, such as personal computers Information from selected web sites is organized into “channels” and provided to a multicast network for multicast transmission to the receivers. Updated channel information is also periodically provided. The receivers store the received channel such that a user can access the web page content in the channel at hard disk speed. Preferably, a conditional access system ensures that only authorized receivers receive the channels. The present invention also preferably includes a lower speed two-way connection to the Internet (such as dial up modem) which is used to report usage information and/or subscription information back to the web sites The present invention also provides “seamless” or automatic access to this connection to allow the user to retrieve any information that has not been received and stored. The receiver also manages use of memory space and other applications that may be active on the receiver to ensure that the receipt and processing of the multicast information does not interfere with receiver operation.

BACKGROUND OF THE INVENTION

[0001] 1. Field Of The Invention

[0002] The present invention relates generally to the distribution ofmultimedia content in a multicast network environment, and moreparticularly to a multicast network system having a high-speed multicastcommunication channel to multicast information to one or more receivers,and also having a return channel, which may operate at a lower speed,for permitting the receivers to interact with the network.

[0003] 2. Description Of Related Art

[0004] The most popular method for distributing multimedia informationis the Internet's world wide web (WWW). Referring to FIG. 1, the WWW canbe considered as a set of network accessible information resources,wherein many web servers 10 and web browsers 12 are connected to theInternet 14 via the TCP/IP protocols. (These protocols are described inthe book “Internetworking with TCP/IP, Vol. I” by Douglas Comer,published by Prentice-Hall in 1991, which is incorporated by referenceherein.) The web browsers 12 typically reside in personal computers(PCs) 16 which are connected to the Internet The connection between thePCs 16 and the Internet 14 is often a low speed connection, such as adial-up modem telephone line connection. The web servers 10 are alsoconnected to the Internet, typically by high-speed dedicated circuitssuch as a 1.5 Mbps T1 connection A PC user uses the browser 12 to accessweb sites 18 (which contain web pages, graphics and other multimediacontent) from the servers 1I via the Internet 14 using HypertextTransfer Protocol (HTTP). This “conventional” method for retrievinginformation from the world wide web requires a separate TCP connectioneach time a user accesses a web site, even if the user repeatedly accessthe same web site.

[0005] The world wide web is founded on three basic ideas:

[0006] (1) a global naming scheme for resources—Uniform ResourceLocators (URLs);

[0007] (2) protocols for accessing named resources—the most common isthe HTTP, and

[0008] (3) hypertext—the ability to embed links to other resources whichis typically done according to the Hypertext Markup Language (HTML).

[0009] Each web site 18 contains a collection of web pages operated by asingle enterprise which appears to a user of a web browser 12 as asingle set of related content. Web pages within a web site 18 areformatted according to the Hypertext Markup Language (HTML) standard.The HTML standard provides for the display of high-quality text,including control over the location, size and font for the text and thedisplay of graphics within the web page. The HTML standard also providesfor the “linking” from one web page to another, including linkingbetween web pages stored on different web servers and even different websites. Each HTML document, graphic image, video clip or other individualpiece of content is identified by an Internet address, referred to as aUniform Resource Locator (URL). As used herein, “URL” refers to anaddress of an individual piece of web content (HTML document, image,sound-clip, video-clip, etc. ), and “URL data item” refers to theindividual piece of content addressed by the URL.

[0010] While very popular, the above-described conventional dial-upmethod of accessing multimedia information is limited in at least twovery important ways. First, most PC users access the Internet usingdialup modems through an ordinary telephone line. These lines operate ata relatively low speed (e.g. 28.8 or 56 kbps) so that the display of anordinary web page (e.g. 150 kbytes) takes a long time (e.g. 50 seconds)and the display of even short video clips (such as a 6 MB movie trailerin low-resolution “Quicktime” format) takes much longer (e.g half anhour). Also, a user's telephone line is unavailable for normal voicecalls the entire time that the Internet is being accessed.

[0011] Second, the conventional method uses point-to-point transfer,wherein each web site 18 must individually deliver its content to eachweb browser 12. Thus, a single 150 kbyte web page must be individuallycarried from the web server across the Internet to each browser thatdisplays that page If a popular web server delivers ten reasonably largepages (for a total of 1 5 MB) to each of 10,000 users during a busyhour, the web server would require at least a 33 Mbps bandwidth link tothe Internet. The link to the Internet and a web server fast enough tofill the link is presently prohibitively expensive and complicatedSupport for a million or more users during the busy hour would becompletely impossible with current Internet technology.

[0012] The world wide web presently supports two methods (advertisementsand subscriptions) for a web site operator to obtain revenue for itssite's content. Advertisements are embedded into a web site's web pages,typically in the form of images, wherein a user can “link” to theadvertiser's web site for more information by clicking on the image.Web-based advertising is superior to normal multicast advertising (e.g.TV, radio and newspaper advertising) in that the web server 10 is ableto track exactly how many users have seen a given advertisement and, forrepeat users, track by user which advertisements and how many suchadvertisements the user has seen.

[0013] If a web site generates revenue through subscriptions a user isonly able to access that web site's content if they have “subscribed” tothe site, i.e, agreed to pay to access the site. By requiring the userto provide an account name and password each time the user wishes toaccess the site, a web site controls access so that only payingsubscribers can access the site.

[0014] Multicast systems are able to accommodate large numbers of usersmuch more easily than the Internet when the users are accessing commoncontent, because a given data item is multicast (i e., sent only once)regardless of the number of receivers. Multicast networks distributingmultimedia content have been in use in recent years over wide-areanetworks, such as a geosynchronous satellite multicast or FM-radioside-band transmission.

[0015] Some multimedia multicast systems have been proposed in academiawhich involve individually multicasting frequently accessed URLs whichhave been tagged to allow the receiver to filter and store only thoseweb pages which, based on past history, may be of interest to thereceiver. These systems have failed to be commercially deployed becausethey copy and multicast content without permission, thereby creatingpossibly conflict with copyright laws. Further, they do not guaranteethe user a consistent set of content which can be accessed “offline”(i.e. not while connected to the Internet) and they do not provide amechanism for preserving a web site's subscription and. advertisingrevenue.

[0016] A commercially deployed multimedia multicast system is AirMedia,Inc.'s Internet Antenna system which uses an FM-radio side-bandmulticast to distribute news and information (with limited graphics). Acomputer terminal receives the multicast information and stores it onthe computer's hard disk. The information is multicast in a proprietaryformat and is made available to the user via a special purposeapplication. The AirMedia system suffers from a lack of compellingcontent, partly due to its very low speed (e.g. 19 kbps) FM side-bandmulticast transmission system Another such system is Intel's Intercastsystem, which uses the vertical blanking interval within an NTSCmulticast to multicast information in order to “data enhance” the TVchannel it is carried on. The vertical blanking interval, which isnormally used to carry closed caption information, is also a low-speedmulticast (e.g. 30 kbps). Unlike the Air Media system, Intercastmulticasts its data in a standard HTML format Intercast provides aviewer application similar to a web browser which, together with specialhardware, allows the user to watch the TV program on a computer monitorwhile simultaneously accessing the supplementary HTML multimedia data.

[0017] The primary problem with such multicast systems is theavailability of good content for multicast broadcasting The providers ofgood content will only specifically prepare their content for amulticast system after there are already a large number of users toreceive such content or when the multicast system operator is willing topay large amounts of money. However, a multicast system operatortypically cannot get a large number of users until the system has goodcontent and cannot obtain the financing to pay for good content until alarge number of users have subscribed to the system This “chicken andegg” problem of good content/number of users has long plagued thedevelopment of multimedia multicast systems.

[0018] In many cases, a new multicast system is launched by recyclingpre-existing content. Television, for example, overcame the “chicken andthe egg” problem by recycling content originally developed for othermedia. Radio shows were converted to be both seen (on TV) and heard,often retaining the original actors, characters, plots and even much ofthe preexisting scripts. Movies, plays, operas and other existingmaterial were also multicast on television. Once television wasestablished using this preexisting content and achieved a critical massof users, television multicasters could afford to develop new contenttargeted exclusively for television (i.e., sitcoms, mini-series, nightlynews, late night talk-shows, infomercials, etc.).

[0019] Another major obstacle to the successful deployment of multicastsystems has been the impact of processing (receiving, filtering,storing, etc) the multicast data on the receiving computer. Priormulticast systems have failed because receiving the multicast datanegatively impacted the receiving computers' performances, such thatusers frequently disabled the multicast application in order to runother applications Often, users would forget to restore the multicastreceiver application, thereby triggering a complete loss offunctionality. Computer games are a classic example of an applicationthat uses all the resources of a computer and, as such, are negativelyimpacted by the receipt and processing of multicast data Further,personal computer operating systems generally do not provide optimalsharing of resources between competing real-time applications. Thus, thetypical personal computer user will not tolerate loss of performance(such as “jerky” video and graphics and/or unresponsive game control)for the sake of receiving multicast multimedia content.

[0020] In some multicast systems, such as Air Media, the impact on thereceiving computer was insignificant due to the low speed of themulticast link.

[0021] However, a high-speed multicast link is very desirable as itincreases the quantity and quality of the content available to the user.Thus, there remains a need for a high-speed multicast system which iscompatible with existing content and does not interfere with otherprocessing performed by the receiving computer.

SUMMARY OF THE INVENTION

[0022] Prior to the present invention, no multimedia multicast systemwas able to solve the number of users/good content “chicken and the egg”problem facing new multicast systems. The present invention overcomesthis problem by providing access to unmodified, high-quality contentfrom existing web sites in a way which requires no changes to theoperation of the web sites and also preserves web sites' advertising andsubscription revenues. The present invention also utilizes a high-speedlink (such as a satellite link) to provide a large quantity of contentand includes several innovative mechanisms for receiving such contentfrom such high-speed link with minimal impact on the operation of otherapplications within the receiving computer. The present invention isreferred to herein as “DirecPC® WebCast” or “WebCast.” (DirecPC® is aregistered trademark of Hughes Network Systems. The basic DirecPC®system, which generally provides a one-way high speed link for receivinginformation from the Internet but does not utilize a multicast network,is described in co-pending application Ser. Nos. 08/257,670 (filed Jun.8, 1994) and 08/795,505 (filed Feb. 7, 1997), which are assigned to thesame assignee as the present invention)

[0023] Generally, WebCast comprises a multicast network which multicastsselected web site content (called “channels”) onto a receivingcomputer's hard disk, making that content available to the user at “harddisk” speed. As set forth in detail below, a channel is a set of webcontent which a user may be interested in repeatedly accessingPreferably, the web content within the channel is periodically updated.WebCast reports sufficient usage information to the web site (i.e. whatcontent has been accessed by the user) to support advertising-basedrevenues, while requiring no change to the web site's operation.

[0024] The present invention also supports subscription-based revenueweb sites. Each receiving computer receives an Electronic Program Guide(EPG), which supports the promotion and subscription to available website channels. The EPG provides functionality beyond what isaccomplished by static web pages in that it permits easy access tochannels to which a user has subscribed and provides promotionalmaterial for all other available WebCast channels. Such promotionalmaterial is accessed from the EPG's cached content until the usersubscribes or unsubscribes to the channel. At this point, the WebCastsoftware contacts a conditional access system in the multicast networkand performs a transaction to initiate or terminate the subscription.The conditional access system optionally provides subscriptioninformation to the multicast system's billing system for subsequentbilling of the WebCast user and/or reporting back to the channel's website.

[0025] Within the receiving computer, WebCast configures a web browserto display the stored (or “cached”) web site channel content. Whenstarted, a content viewer on the receiving computer configures the webbrowser to access the cached content via an HTTP proxy-server within theWebCast content viewer. A “cache-miss” occurs when the browser requestsa URN which is not stored in the cache of multicast data. When acache-miss occurs, the WebCast software notifies the user and offers theuser a choice of accessing the content via normal (i.e dial-up) Internetaccess. If the user so chooses, the WebCast application establishes aconnection to the Internet (if needed) and forwards the requests forcontent to the web server on the Internet which contains the missingcontent. Preferably, this Internet connection is “seamless,” in that itrequires no action by the user but is automatically established by theWebCast software. WebCast also from time-to-time establishes aconnection to the Internet to send usage information to the appropriateweb sites.

[0026] According to one aspect of the present invention, a system andmethod that transmits content organized into channels, wherein achannel's content includes a plurality of URL data items and each URLdata item is addressed by a URL, comprises assigning one or moremulticast addresses to each channel, scheduling the assembling of achannel's content, assembling the channel's content, fragmenting thechannel's content into packets, wherein each packet is addressed withone of the channel's multicast addresses, and multicasting the packets.

[0027] According to another aspect of the present invention, a systemand method that transmits content organized into channels, wherein achannel's content includes a plurality of URL data items and each URLdata item is addressed by a URL, comprises scheduling the assembling ofa channel's content, assembling the channel's content, compressing asubset of the URL data items, wherein each URL data item is compressedindividually independent of other URL data items such that eachcompressed URL data item can be decompressed without decompressing otherURL data items, fragmenting the channel's content into packets, andmulticasting the packets. The present invention may further assemble abase package of the channel's content, wherein the base package containseach URL data item in the channel, and assemble a delta package of thechannel's content, wherein the delta package contains URL data itemswhich have changed or are new since the previous assembling of the basepackage.

[0028] According to yet another aspect of the present invention, asystem and method that transmits content organized into channels,wherein a channel's content includes a plurality of URL data items andeach URL data item is addressed by a URL, comprises scheduling theassembling of a channel's content, assembling the channel's contentaccording to the schedule, fragmenting the channel's content intopackets, multicasting the packets to a plurality of receivers, whereineach receiver stores the received channel's content in a receivermemory, and receiving usage reports from each receiver, wherein eachusage report identifies a subset of URL data items from the stored URLdata items that were accessed from the receiver memory.

[0029] According to yet another aspect of the present invention, areceiver for receiving from a multicast network content organized intochannels, wherein a channel's content includes a plurality of URL dataitems and each URL data item is addressed by a URL, and wherein themulticast network transmits the channel's content to the receiver inpackets, determines a multicast address used to carry a channel'spackets, enables reception of packets containing a channel's multicastaddress, receives the packets containing a channel's multicast address,assembles the received packets into a channel's content, stores thechannel's content, and allows a user to access the stored channel'scontent. The receiver may further individually decompress eachcompressed URL data item in the stored channel content at a time whenthe user accesses the URL data item.

[0030] According to yet another aspect of the present invention, areceiver in a multicast system receives URL data items from a multicastnetwork, stores the received URL data items, allows a user to access thestored URL data items, and tracks user access to the stored URL dataitems. The receiver may further determine when a URL data item requestedto be accessed by the user is not present within the stored URL dataitems, notify the user that the requested URL data item is not stored,and allow the user to access the non-stored URL data item via aconnection (such as dial-up modem) to a TCP/IP network, such as theInternet.

[0031] According to yet another aspect of the present invention, areceiver in a multicast system monitors receiver activity, andselectively receives content from a multicast network, wherein thecontent is selectively received based on the monitored receiveractivity. The receiver may be, for example, a personal computer and themonitored activity may include other applications/programs running onthe receiver, disk/memory utilization and/or user input (keystrokes ormouse clicks). The receiver may further suspend reception of contentpending conclusion of the monitored activity, such that reception doesnot interfere with the monitored activity.

[0032] According to yet another aspect of the present invention, areceiver in a multicast system comprises a package receiver forreceiving packets containing URL data items from a multicast network andassembling the received packets into a channel, wherein the channelcomprises a set of URL data items, a memory for storing the channel, anda content viewer for allowing a user to request access to and access theURL data items in the stored channel.

[0033] According to yet another aspect of the present invention, asystem for multicasting URL data items from web sites to a plurality ofreceivers comprises a web crawler for retrieving URL data item from thethe web sites and formatting the retrieved URL data item into packages,a package delivery subsystem for receiving the packages from the webcrawler, fragmenting the packages into packets and transmitting thepackets to a multicast network, and a conditional access system fordetermining which receivers are authorized to receive the packets,wherein the multicast network multicasts the packets only to authorizedreceivers.

[0034] According to still another aspect of the present invention, asystem for multicasting content organized into channels to a pluralityof receivers, wherein a channel's content includes a plurality of URLdata items from at least one web site, comprises a web crawler forretrieving the URL data items from the web site via a TCP/IP network andformatting the retrieved URL data items into packages, a packagedelivery subsystem for receiving the packages from the web crawler andfragmenting the packages into packets, a conditional access system fordetermining which receivers are authorized to receive the packets, and amulticast network for receiving the packets from the package deliverysubsystem. The conditional access system encrypts the packets and themulticast network multicasts the encrypted packets to the authorizedreceivers, wherein the authorized receivers store the packets in amemory and decrypt the packets

BRIEF DESCRIPTION OF THE DRAWINGS

[0035]FIG. 1 is a simplified block diagram of a prior art method foraccessing information from the Internet;

[0036]FIG. 2 is a block diagram of the multicast system of the presentinvention;

[0037]FIG. 3 is a block diagram of the back-end subsystem of themulticast system of FIG. 2;

[0038]FIG. 4 is a block diagram of the process used by the web crawlerof FIG. 3 to gather active web site content,

[0039]FIG. 5 is a flowchart illustrating the steps performed by thepackage delivery subsystem of the back-end subsystem of FIG. 3;

[0040]FIG. 6 is an example window of an Electronic Program Guide (EPG)that may be used to notify a user of the maximum memory space requiredfor a channel and of the package broadcast schedule;

[0041]FIG. 7 is an example window of the Electronic Program Guide (EPG)which allows a user to preview the content of available channels andpermits the user to subscribe or unsubscribe to the channels;

[0042]FIG. 8 is a flowchart illustrating the steps performed by thecontent viewer of the receiver of FIG. 2 when a “cache miss” occurs;

[0043]FIG. 9 is an example of a dialog box generated by the contentviewer to notify the user of a cache miss and query the user whether aconnection to the Internet should be established,

[0044]FIG. 10 is a block diagram illustrating the usage reportingfunction of the present invention;

[0045]FIG. 11 is a flowchart illustrating the steps performed by thecontent viewer of FIG. 7 to report usage information; and

[0046]FIG. 12 is an example of a dialog box generated by the contentviewer to request permission from a user to connect to the Internet toreport usage information

DETAILED DESCRIPTION

[0047] WebCast Channels

[0048] The present invention organizes the URL data items it transmitsinto “channels,” wherein a channel is a set of URL data items which auser may be interested in repeatedly accessing. A channel ordinarily isa subset of a web site's content (i.e. a set of web pages) to beperiodically extracted from the web site by a web crawler and deliveredto subscribing users by conditional access protected multicast filetransfer. Thus, a channel's content consists of a collection of URL dataitems, typically all from a single web site. Preferably, a channel'scontent is periodically updated. A typical channel might, for example,contain the content of 3,000 different URL data items. Examples mightinclude any web site, such as www.direcpc.com or www.hns.com, a generalnews web site, such as www.abcnews.com, or a financial news site, suchas quicken.excite.com.

[0049] A user subscribes to WebCast channels of interest and onlysubscribed channels are received and stored on the user's receivingterminal. A channel's content and multicast schedule is specified by aWebCast channel definition. Each channel's channel definition ispredetermined However, WebCast may allow a channel definition to bealtered. Each channel definition includes at least:

[0050] (1) A list of web strands, wherein each strand includes astarting address (URL) and a search depth Since each web page maycontain links to other pages, search depth refers to the number of linksthat are to be retrieved and stored with the original page In apreferred embodiment, the search depth is set to two “levels” from anoriginal web page Thus, the channel will include the original web pageand all other pages which are two hypertext links (i.e. two mouseclicks) away from the original page The search depth, however, may beset to any number of levels as best suits the web site to be included inthe channel.

[0051] (2) A list of filter settings indicating which URLs should beincluded in or excluded from the channel. Another set of filters mayalso identify how, if at all, URL data item is to be compressed prior tobeing transmitted. Yet another set of filters may define the“hit-tracking” attributes to be assigned to each URL, as discussed indetail below. The web crawler starts at each of the starting URLs andsearches all links that pass the filters to the specified search depth.

[0052] (3) A schedule specifying how often and in what way a channel'scontent is packaged and multicast by a multicast network to each user.

[0053] System Components

[0054] Referring to FIG. 2, the WebCast system 20 of the presentinvention consists of a back-end subsystem 22 which communicates withone or more multicast networks 24 (link C). The back-end subsystem 22 isconnected to a plurality of web sites 18 (from which content isgathered) via a TCP/IP internetwork, such as the Internet 14 (links A,B) The multicast network 24 multicasts information retrieved from theweb sites 18 to a plurality of receivers 26 over a high-speed link (F),such as a satellite or other high-speed (over 200 kbps) link. Eachreceiver 26 may be, for example, a personal computer in a user's home orbusiness. However, the receivers 26 may also comprise set top boxes,digital televisions or other devices capable of receiving Internetcontent. Each receiver 26 is also preferably connected to the Internet14 by a low-speed link (D), which may be, for example, dial-up modem,ISDN, two-way cable, or the like. Further, the present invention couldbe implemented with other TCP/IP networks other than the Internet, suchas intranets.

[0055] The basic functions of the WebCast system components are setforth below and the various data flow paths are depicted in FIG. 2. Itis understood, however, that the described functions may shift locationamong the system's components depending on a particular systemconfiguration

[0056] Back-End Subsystem Functions

[0057] The back-end subsystem 22 (which generally comprises a computeror set of networked computers) performs at least the followingfunctions:

[0058] (1) Gathering URL data item from the web sites 18 via theInternet 14 or a private TCP/IP network or intranet (links A and B) andassembling it into packages.

[0059] (2) Fragmenting the large (i.e multi-megabyte) packages (whichcontain URL data items from the web sites 18) into an appropriatesequence of packets and passing the packets to the multicast network 24(link C);

[0060] (3) Optionally processing usage reports from the receivers 26(links D and A) and providing the usage reports back to the web sites 18(links A and B),

[0061] (4) Optionally processing channel subscription and unsubscriptionrequests from the receivers 26 (links D and A) and optionally forwardingsuch requests to the web sites 18 (links A and B); and

[0062] (5) Optionally performing conditional access for the WebCastchannels. Conditional access is normally performed by a conditionalaccess system 25 in the multicast network 24, but can be implemented bythe back-end subsystem 22 if the multicast network does not include aconditional access system or if use of the multicast network'sconditional access system is not desirable.

[0063] Multicast Network Functions:

[0064] The multicast network 24 performs at least the followingfunctions:

[0065] (1) Receiving packets from the back-end subsystem 22 (link C) andoptionally multiplexing the packets with data (such as digital video,audio voice, etc.) from any other broadcast source(s) 27 (link E);

[0066] (2) Multicasting the packets to the receivers 26 over thehigh-speed link (link F);

[0067] (3) Optionally performing conditional access via the conditionalaccess system 25 to ensure that only subscribing receivers may receive achannel's packages,

[0068] (4) Optionally providing a return path from the receivers 26 tothe back-end subsystem 22 to allow usage reporting;

[0069] (5) Optionally processing channel subscription and unsubscriptionrequests from the receivers 26; and

[0070] (6) Optionally providing Internet access to allow a receiver toaccess “cache-misses” from the Internet.

[0071] Receiver Functions:

[0072] Each receiver 26 performs at least the following functions:

[0073] (1) Interacting with the multicast network 24 to enable receptionof the appropriate addresses;

[0074] (2) Processing received packets from the multicast network 24(link F);

[0075] (3) Reassembling WebCast channel packages from the receivedpackets and storing each as a file in a memory (e.g. hard disk) 28;

[0076] (4) Managing space in the memory 28 and managing use of thereceiver's resources to minimize impact on other applications running onthe receiver when multicast packages are received and processed;

[0077] (5) Providing the user with promotional material (i.e. throughthe EPG) that helps the user determine which WebCast channels tosubscribe to,

[0078] (6) Providing the multicast network 24 (or optionally theback-end subsystem 22) with subscription or unsubscription requests;

[0079] (7) Reporting usage information to the back-end subsystem 22 viathe multicast network;

[0080] (8) Optionally decrypting the received packages when themulticast network is not providing conditional access but conditionalaccess is desired, and

[0081] (9) Interacting with the back-end subsystem 22 to obtainconditional access key material if the back-end subsystem (rather thanthe multicast network) is performing conditional access.

[0082] Back-End Subsystem Components

[0083] Referring to FIG. 3, the back-end subsystem 22 includes two majorcomponents. (1) one or more web crawlers 30, and (2) a package deliverysubsystem 36. Each web crawler 30, generally, is a computer whichaccesses channels from the web sites 18 according to predeterminedchannel definitions 32. The web crawler 30 is similar to commerciallyavailable web crawlers, such as TelePort Pro and WebWacker. The primarydifference from commercial web crawlers is that commercial web crawlersstore each URL data item as a file on disk and the DirecPC WebCast webcrawler 30 formats the URL data items into packages (see below). Asexplained in detail below, the web crawler 30 gathers URL data item fromthe web sites 18. The URL data item is gathered from the list of URLs inthe channel definition and other URLs which are “linked” to the listedURLs according to the search depth. The web crawler 30 then formats thegathered URL data item into packages 34 (also explained in detail below)and submits the packages 34 to the package delivery subsystem 36.

[0084] The package delivery subsystem 36 receives the packages 34 fromthe web crawler(s) 30 and fragments the packages 34 into an appropriatesequence of multicast packets 38, which are provided to the multicastnetwork 24 for multicast transmission to the receivers 26

[0085] The back-end subsystem 22 also optionally includes one or morecache hit trackers 40 which receive usage reports 42 from the receivers26. The usage reports may be stored as hit log files 44 which areperiodically delivered to the web sites 18. (The usage reportingfunction of the present invention is described in detail below.)

[0086] The back-end subsystem 22 also optionally includes a registrationserver 46 (which may also be referred to as an auto-commissioningserver). The registration server 46 provides a convenient method forusers to subscribe to WebCast channels and also produce package deliveryenvelopes and subscription billing records 48 which track subscriptionsand canceled subscriptions. (The subscription function of the presentinvention is also described in detail below)

[0087] Web Crawling

[0088] As set forth above, the back end subsystem 22 contains one ormore web crawlers 30 which package a channel's content and submit it tothe package delivery subsystem 36 for multicast transmission to thereceivers 26 via the multicast network 24 The process whereby the webcrawler(s) 30 gather content from various web sites 18 is generallyreferred to as “crawling.” Preferably, the web crawlers 30 crawl the websites periodically and/or on a scheduled basis. The web crawler 30formats a channel's content into a single data structure that ispreferably stored and transferred as a computer “stream” or “flat” file(which are used by most personal computer file systems. A filecontaining channel content is referred to as a package 34.)

[0089] The web crawler 30 may be located in the back-end subsystem 22near the package delivery system 36 which, in some cases, minimizes theeffort to manage web crawling. Alternatively, the web crawler 30 may belocated near the web site(s) it is crawling, for example, on the samelocal area network as the web site's servers. This configuration reducesthe amount of traffic across the wide area network in that onlycompressed packages (rather than all of the channel's content) are sentacross the wide area network.

[0090] The web crawler 30 typically does not receive an exhaustive listof the URLs to be included in the package. Instead, the web crawlerreceives a channel definition 32 (which may either reside in the webcrawler or be retrieved by the crawler from an external server)containing a list of the URL addresses of the URL data items to beincluded in the channel. For each starting URL address in the channeldefinition 32, the web crawler 30 creates a list of URL addresses to becrawled, which initially contains only the starting address The webcrawler 30 repeatedly retrieves each U-L data item and removes that URLaddress from its list until the list is empty. The web crawler mayperform many such retrievals in parallel to reduce the time needed tocrawl a channel.

[0091] The web crawler 30 analyzes the content of these linked dataitems of each retrieved URL data item to determine if it contains“links” to other URL data items. Typically, only HTML pages containlinks. If so, the URL addresses are added to the list of URL addressesto be crawled if the additional URL data item passes the channeldefinition filters. The URLs added to the list may include frames,embedded graphics, java applet classes and references (also called“links”) to other URLs external to the page. Frames can be thought of asembedded HTML pages. Frames, embedded graphics, java applet classes andembedded graphics within the frames can be thought of as part of a webpage and are at the same “level” or “depth” as the HTML which referencedthem. URLs which are “linked” from the page under analysis areconsidered one level “deeper” (i.e, further away from the startingaddress) than the page that referenced them. A URL's crawl depth isstored, along with the URL address itself, on the list of URLs to be“crawled.” A URL is only placed on the list of URLs to be crawled if itpasses the channel definition's filters and if its depth does not exceedthe search depth associated with its starting address and the URL hasnot previously been entered into the list.

[0092] A web server will often respond to a request for a URL by“redirecting” the request to another URL (often on a different webserver). Such a redirection does not increase the depth of the URL thatis eventually retrieved

[0093] The web crawler(s) 30 can be programmed to gather the entire setof a channel's content and place it into a package 34, which is referredto herein as a “base” package Web crawling to create a base package isreferred to herein as a “base crawl.” A channel's content, however, maybe frequently updated by the web site operator and the updates may occuron an unknown basis and it is important to provide a WebCast user withan updated and consistent representation of a web site's content. Thus,once a base package has been produced, the web crawler 30 can bescheduled to produce a package which contains only the URL data itemswhich have changed since the base crawl occurred. A package containingonly the changed or updated content is referred to herein as a “delta”package. Web crawling to create a delta package is referred to herein asa “delta crawl.”

[0094] Upon retrieving a URL data item, the web crawler determineswhether and how to include the URL data item within the package beingproduced by the web crawler. This determination depends on: (1) variousfilter settings; (2) whether a base or delta package is being crawled;and (3) if a delta package is being crawled, whether the URL data itemis present in the channel's base package and, if so, whether the URLdata item has changed.

[0095] If a base package is being crawled, a URL is included in thepackage if it passes the filters. If a delta package is being crawled,the URL is included in the package if it passes the filters and iteither does not exist in the base package or has changed since theversion in the base package Whether the URL has changed can bedetermined either by actually comparing the content or by checking the“last-modified” field provided by the web server. For some sites, theactual comparison of data is needed as the “last-modified” field cannotbe relied upon. Alternatively, the web crawler 30 can determine whetherthe URL data item has changed by comparing checksum(s) or messagedigest(s) associated with the data item.

[0096] Crawling is typically the preferred mechanism for obtaining achannel's URLs in that it requires no changes to the web site'sproduction and operation. This is very important when overcoming thenumber of users/good content “chicken and egg problem” describedearlier. Crawling may, however, waste networking and web serverresources in that each URL must be individually checked on each crawl.For a 40 MB base package (uncompressed), which is being crawled everyhalf hour, this data flow exceeds 80 kbps.

[0097] Thus, for web sites that correctly set the “last-modified” dateof URLs, the present invention reduces processing time by requesting theURLs with an HTTP “Get If Modified Since” request. Web sites which areavailable for subscription in WebCast produce their own channeldefinitions which accurately indicate both what URLs are to be includedin the channel and when the URLs have changed. Such a channel definitionmay simply list every URL to be included in the channel and when eachwas last modified With such a channel definition, the web crawler, bystoring retrieved URLs and their modification date, need only gatherURLs which are new or which have changed since the previous crawl. Thisprovides a web site with maximum control over the content in itschannel, while minimizing the web server and networking resourcesrequired to gather the content.

[0098] Dynamic web site content poses a special problem to WebCastcrawling in that the simple analysis of HTML is insufficient to gatherall the required URLs for a channel. An example of dynamic content is aSports Score Java Applet which, when started, retrieves and displays aset of sports scores. The URL containing the scores is not referenced bythe HTML and would not normally be gathered by the web crawler. The datacan only be gathered by actively running the Java Applet

[0099] To overcome this deficiency, the channel definition allows a listof pages whose content is gathered by a Java-capable browser. Referringto FIG. 4, the web crawler 30 configures a Java-capable browser 50 touse the web crawler as a proxy-server and directs the browser 50 todisplay the configured pages with dynamic content (data flow A). Thebrowser 50 gathers the active content from a web server 10 via the webcrawler 30, which is now acting as a proxy-server (data flow B)Configuring the browser 50 to use the web crawler as a proxy-serverallows the web crawler to monitor and record the URLs accessed by thebrowser, including the active content (data flow C).

[0100] Compression

[0101] As set forth above, the web crawler 30 gathers URL data items fora channel and places them into packages 34 Each package contains: (1) aset of URL data item; (2) indexing information, such as a hash table, toallow quick access to the URL data item; and (3) various supplementalinformation identifying the set of URL data item contained by thepackage and other information to guide the use of its content. The webcrawler 30 uses a lossless compression algorithm, such as the onecreated by Liv and Zempel in 1977 (LZ77) or other algorithm, toindividually compress each URL data item. (An alternative losslesscompression algorithm is described in co-pending application Ser. No.08/982,864 entitled “Data Compression For Use With A CommunicationsChannel, filed on Dec. 2, 1997 and assigned to the same assignee as thepresent invention) URL data item which can be compressed is individuallycompressed (and flagged as compressed) prior to being placed into apackage 34 (rather than compressing the package as a whole).

[0102] In many cases, URL data item which has changed between the timeof a base crawl and a delta crawl has only partially changed. Forexample, many web sites change the advertisements embedded into a HTMLpage each time the page is served. In such a case, only the fewcharacters containing the URL data item of the advertisement change fromone crawl to the next crawl. In other cases, the bulk of a web pageconsists of “boiler plate” which provides a consistent look and style tothe web page Even when the real content within such a web page hascompletely changed, only a small fraction the page's characters change.In view of this, the size of delta packages can be minimized if the webcrawler 30 performs what is referred to herein as “differencecompression.”

[0103] With difference compression, the web crawler 30 compares a URLdata item to be included in a delta package with the corresponding URLdata item in the base package (if it exists). The web crawler 30 dividesthe delta package URL data item into sections of data and, for eachsection, places into the compressed version of the URL data item either:

[0104] (1) a reference to the base package's URL data item where thatsection of data can be found because it appears identically in both thebase package's and the delta package's URL data item This reference isgenerally much smaller than the data itself, which inherently providescompression. An example of a reference is an offset from the beginningof the URL to the first byte and an offset to the last byte beingreferenced. Other more complex, but more compact mechanisms mayalternatively be used to encode a reference; or

[0105] (2) the section of data from the delta package's URL data item.This does not provide any compression but exists to ensure that thedelta package's URL data item can be reassembled without modificationfrom the base package's URL data item and the difference compressed URLdata item. The difference compressed URL data item can then beoptionally compressed with a lossless data compression algorithm, suchas LZ77, prior to being placed into the delta package.

[0106] The channel definition filters indicate whether a URL data itemincluded in the package should be compressed and, if so, the compressionalgorithm to be used. For delta packages, the channel definition filtersalso specify whether difference compression is to be applied (and whichalgorithm to use to determine the difference) to a URL to be included inthe delta package.

[0107] Table 1 provides examples from WebCast channels illustrating theeffectiveness of base/delta packages, individual compression of URL dataitems and difference compression. TABLE 1 ESPN WebCast Channel ABC NewsSportszone Size of base package without compression 38.1 MB   22 MB Sizeof base package with standard LZ77 19.8 MB 10.2 MB compressionCompression ratio with standard LZ77 1.92 to 1 2.16 to 1 compressionSize of delta package with LZ77 11.8 MB  6.3 MIB compression only (i.e.no difference compression) Size of delta package with LZ77  2.8 MB  1.6MB compression and difference compression Compression ratio with deltapackage, 13.61 to 1 13.75 to 1 LZ77 compression and differencecompression

[0108] The resulting compression ratio (i.e. approximately 14 to 1) ismuch higher than compression ratios achievable from the directapplication of ordinary lossless data compression algorithms, whichtypically do not exceed 2.5 to 1 on such packages. The use of deltapackages in combination with base packages thus greatly reduces theamount of bandwidth needed to keep a receiver “up-to-date” with a website compared to repeatedly multicasting the full set of a web site'scontent. The present invention also achieves this while preserving theobjective of always presenting a consistent snapshot of a web site'scontent.

[0109] The present invention may also use delta packages of deltapackages For example, a sports web site may frequently update scores fora game with no other changes to the web site. In such a case, only theURL data item containing the scores changes from one crawl to the next.Thus, a delta package may be generated and difference compressed suchthat it contains only the difference (i.e. the new score) from theprevious delta package. The use of delta packages of delta packagesfurther reduces the amount of data which must be transmitted.

[0110] Advantages of Packaging and Compression

[0111] The present invention's organization of web site content intochannels and packages provides significant advantages and benefits forreducing the impact of multicast receiving on the function of otherapplications active on a receiver 26. For example, organizing eachpackage 34 as a single file with built-in indexing for quick accessminimizes the processing that the receiver 16 must perform on thecontent prior to displaying it to the user. The alternative, which is tostore each URL data item as a separate file (as is commonly done withexisting browser caches), requires significant processing (andadditional memory/disk space) to divide the URL data items intoindividual files and to delete obsolete files. Because many web channelsmay contain over 3000 URLs, this processing (i.e. of 3000 separatefiles) may take several minutes or longer. Further, this processing musttake place either after the content has been received (disruptingwhatever other processing is taking place at the time) or when the userfirst accesses the content (causing the user to have to wait while thisprocess is taking place). In addition, retaining all the URL data itemin a single file reduces the processing required to display content byeliminating the overhead of opening and closing a file for each URL.

[0112] Compressing each URL data item individually (rather thancompressing the package as a whole) also minimizes the processing thatthe receiver 16 must perform prior to displaying a package's content.Decompressing the package as a whole requires significant processingwhich, as set forth above, must take place either upon reception(disrupting whatever is happening at the time) or when the user firstaccesses the package's content (forcing the user to wait). Whole packagecompression also uses more memory or disk space because the content ofthe whole package must be stored in decompressed form. Decompressingeach URL data item individually as it is needed allows the data items tobe stored in compressed form (reducing memory space) Also, only dataitems that are actually accessed by the user is decompressed, thusreducing overall processing time. Also, because the decompression of URLdata items is performed at a time when the user has directed thereceiver to display content, the decompression is not likely to disruptother applications that require use of receiver resources

[0113] Still further, the use of delta and base packages allows receiptof base packages to be scheduled for periods of time when the receiver16 can be expected to be idle (e.g. late at night) while the smallerdelta packages can be received throughout the day. Thus, a receiver canbe powered off, or can suspend, abort or terminate package receptionduring times when the receiver is dedicated to other processing, yetstill be quickly brought “up-to-date” after package reception has beenenabled. The present invention also ensures that the user is presentedwith a complete, consistent version of the content. Thus, the presentinvention provides significant advantages over other multicast systemswhich often required the receiver to be powered up continuously suchthat users did not use the system or accidently terminated reception byturning of the receiver. With delta and base package transmission,particularly where base packages are transmitted overnight or duringsome other inactive period, a user can turn off the receiver or suspendpackage reception during peak usage time and still have the receiver bequickly brought up to date when package reception is restarted.

[0114] The use of delta packages, particularly with differencecompression, results in much smaller transmissions because less datamust be received and processed to keep a receiver up-to-date. Thisresults in lower bandwidth requirements and lowers the impact on thereceiver when a delta package is received.

[0115] The organization of content into channels which are subscribed toby the user and into single-file packages further allows the content tobe transmitted via very efficient multicast file transfer protocolswherein virtually no receiver processing is wasted on filtering outcontent which is not of interest to the user. It also allows thescheduling of transmissions such that a user can determine whenreception must be enabled to receive the content they desire.

[0116] While the preferred embodiment of the present invention storeseach base or delta package as a single file, storing each package in asmall number of files (i.e. wherein the number of files is less than thenumber of URLs in the package) still achieves many of the benefitsdescribed above and may be desirable within some multicast networks. Inaddition, creating the indexing information within the receiver from asingle file (or small number of files) also achieves many of thebenefits described above

[0117] Multicast Network Components

[0118] Referring again to FIG. 2, the multicast network 24 generallyincludes the following components, although some components may varydepending on the specific embodiment of the multicast network:

[0119] (1) A head-end subsystem 52, which is responsible for: (a) takingpackets from the back-end subsystem 12 (link C) and multicasting thepackets to the receivers 26 (link F); and (b) optionally multiplexingthe back-end subsystem's packets with data (i.e. digital video, audio,etc.) from another broadcast source 27 (link E) and multicasting theresulting data stream(s) to the receivers 26.

[0120] (2) A multicast receiver 54, which is responsible for providingmulticast packets from the requested channels to the receivers 26 (linkF). Alternatively, the multicast receiver may be integrated with thereceiver 26, such as in a personal computer with a receiver peripheraland associated software A digital satellite TV settop box or a cable TVsettop box are other examples of receivers that may include integratedmulticast receivers.

[0121] (3) A conditional access system 25, which may be integrated withthe head end subsystem 52 The conditional access system 25 may be of thetype described in U.S. Pat. Nos. 5,481,609; 5,282,249, 5,659,615; or5,652,795, which are incorporated herein by reference. General cryptotechnology is also described in “Applied Cryptography, 2nd Edition”,published by John Wiley and Sons, 1996, which is also incorporated byreference.

[0122] The conditional access system 25 uses a cryptographic key or setof keys to encrypt packets in such a way that a receiver 26 onlydecrypts packets which it is authorized to receive In general, thereceiver 26 notifies the multicast receiver 54 of the channels to bereceived. If the multicast receiver 54 is integrated with the receiver26, this is done through a software interface The multicast receiver 54then contacts the conditional access system 25 to start encrypting thepackets and the receiver is provided with the appropriate key to decryptthe packets which it is authorized to receiver.

[0123] In the context of this invention, the multicast network is notlimited to any particular type of network and may comprise any digitalmulticast network wherein the data being carried is segmented and placedinto one or more relatively small packets (e.g. approx. <1 MB) and whereeach packet includes a multicast address field. In a preferredembodiment, the multicast network 24 is a geosynchronous satellitedirect to home system carrying both digital video and data services. Inanother embodiment, the multicast network 24 is a two-way cabletelevision network carrying both analog (and/or digital) television aswell as digital multicast data services. The multicast network may alsocomprise any other type of multicast network, such as ethernet (whereinthe destination MAC address field carries the multicast address),Digital Video Broadcast (DVB) satellite, terrestrial and other media,ATSC digital television and other multicast networks that use MPEG2transport packets and wherein the PID field carries the multicastaddress, MBONE (experimental Internet multicast network), DVB carryingmultiprotocol encapsulated data, and any other IP (Internet Protocol)multicast network wherein IP packets are used and the destination IPaddress field holds a multicast address

[0124] Receiver Components

[0125] In a preferred embodiment, the receiver 26 is a personalcomputer. However, the receiver 26 may comprise any component capable ofreceiving and processing packets from a multicast network, such as asettop box which provides data services (optionally along with digitalvideo services) which are viewed through a television, or a digitaltelevision which integrates the functions of a settop box and atelevision to provide both video and data services. Portable or handheldcomputers and the like with wireless receivers are another example of areceiver that may be used with the present invention.

[0126] Referring to FIG. 2, each receiver 26 includes software which maybe functionally decomposed into the following components. It isunderstood, however, that the actual organization of the software mayvary within a specific implementation.

[0127] (1) A Package Receiver 56—The package receiver 56 processesreceived packets from subscribed-to channels via the multicast receiver54 and reassembles the packages from those packets. (As set forth above,the multicast receiver may also be located in the receiver.) Asdiscussed above, each package is stored as an individual file (or asmall number of files) in the receiver memory 28 (i.e. on a receivingcomputer's hard disk). Optionally, if the multicast network 34 is notproviding conditional access, the package receiver 56 may decrypt thepackages, if conditional access is desired. The package receiver alsomanages the use of space in the receiver memory 28 and manages the useof receiver resources to minimize the impact of multicast receiving andprocessing on other applications running on the receiver.

[0128] (2) A Content Viewer 58—The content viewer 58 in the receiver 26provides the user with the promotional material (i e through theElectronic Program Guide (EPG)) that helps the user determine whichchannels to subscribe to. In a preferred embodiment, the content vieweralso interacts with the multicast network 24 (or optionally the back-endsubsystem 22) to subscribe or unsubscribe to channels at the user'sdirection. If the back-end subsystem 22 is performing conditional access(rather than the multicast network), the content viewer 58 preferablyalso interacts with the back-end subsystem to obtain key material Thecontent viewer 58 may also preferably report usage information (i.e.information about which channels were accessed by the user) to theback-end subsystem 22.

[0129] Package Transmission

[0130] In the preferred embodiment, package transmission takes placeover a conditional access controlled multicast network 24 which carriesIP multicast packets and where each channel is assigned an IP multicastaddress. The multicast network's conditional access system 25 ensuresthat only subscribed receivers may access a channel's IP packets.Preferably, the multicast network 24 provides a single IP multicastaddress (which every receiver 26 may receive) on which the packagedelivery subsystem 36 sends packets announcing the upcoming transmissionof packages.

[0131] In the preferred embodiment, the receiver 26 may selectivelyenable and disable IP multicast addresses. If an IP multicast address isdisabled, the multicast receiver 54 filters packets containing theaddress so that the receiver 26 is not burdened with the processingassociated with identifying and discarding disabled addresses.

[0132] Alternatively, if the multicast network 24 is compliant with theEuropean Digital Video Multicast (DVB) standard, each channel may beassigned an MPEG2 transport stream and the multicast network'sconditional access system 25 ensures that only subscribed receivers mayaccess a channel's MPEG2 transport stream

[0133] In other embodiments, more than one multicast address may bededicated to each channel (e.g. one address for base packagetransmissions, one for delta package transmissions, one for controlinformation, etc.). An address or set of addresses may also be shared bymultiple channels, one address or set of addresses per package at atime. These arrangements, however, are not preferred as they complicatethe conditional access system 25.

[0134] Preferably, the package delivery subsystem 36 in the back-endsubsystem 22 receives the packages 34 from the web crawler(s) 30 (FIG.3) with a schedule for each package. The schedule includes informationsuch as when transmission should take place, the priority of thepackage's transmission, the speed at which the transmission should takeplace, whether the package should be transmitted more than once, and/orother information related to package transmission

[0135]FIG. 5 is a flowchart illustrating the steps performed by thepackage delivery subsystem 36 of the back-end subsystem 22 to transmit apackage 34 to the multicast network 24. If a schedule is provided withthe package, the steps are performed within any schedule constraints.The package delivery subsystem (PDS) 36 first formats a multicastannouncement packet identifying the package to be transmitted (block60). The multicast announcement packet may include, for example, whichchannel the package is part of, what kind of package is beingtransmitted, information to uniquely identify one version of a packagefrom another, and the size of the package. A PDS 36 then passes themulticast announcement packet to the multicast network 24 to betransmitted over the multicast announcement address (block 62).Preferably, the PDS 36 transmits the packet multiple times to ensure ahigh probability of reception.

[0136] The PDS 36 then waits a small period of time (e.g. 3 seconds) toallow the receivers 26 to determine whether the package should bereceived and to prepare for reception (block 64). Optionally, ifconditional access is desired but is not provided by the multicastnetwork 24, the PDS 36 uses a key or set of keys only made available tosubscribed receivers to encrypt either the entire package or theindividual packets (block 66)

[0137] The PDS 36 then fragments the package into a sequence of packets,wherein each packet has a unique sequence number (block 68) and beginstransmitting the packets at the specified bit rate (block 70). Thepackage delivery subsystem 36 also multicasts other information (eitherwithin the multicast announcement packets or other packets) to allow thereceiver to identify the first and last packet of a package. The PDS 36may subsequently retransmit a package's packets to increase theprobability of reception. Preferably, the PDS 36 retransmits packages ona scheduled basis and only certain packages may be scheduled forretransmission For example, base packages may preferably be transmitted2-3 times depending on factors such as the time of day and the size ofthe package Delta packages, however, may preferably be transmitted justonce, as packet loss may be recovered by the subsequent transmission ofthe next delta package

[0138] Package Reception

[0139] The package receiver 56 in each receiver 26 may optionally beconfigured to monitor receiver activity and/or user input to classifythe receiver's readiness to receive packages. For example, the packagereceiver 56 might monitor: (i) receiver computer terminal CPU loading;(ii) receiver computer disk activity; (iii) user input (mouse clicksand/or keyboard keystrokes); and/or (iv) time of day. The user may alsoenter preferences of when packages should be received. For example, theuser may specify that prior to starting a computer game, all packagereception should be suspended until at least 30 minutes has gone by withno user input (mouse clicks or keystrokes), or until at least 30 minutesof only minimal CPU loading and disk activity Alternatively, the usermay allow only base package reception under such circumstances.

[0140] Thus, a receiver's readiness for package reception may beclassified as follows:

[0141] (1) Completely suspended—the package receiver 56 will receive nopackages in order to prevent interfering with other processing;

[0142] (2) Base package reception only—the package receiver 56 willreceive only base packages to reduce interference with other processing;or

[0143] (3) Package reception enabled—the package receiver 56 willreceive any package (i.e. base or delta) of interest.

[0144] Preferably, when package reception is completely suspended, thepackage receiver 56 suspends reception of the multicast announcementpackets by disabling their addressees) In addition, the package receivermay reduce its impact by releasing all or part of its resources (memory,threads, etc.) up to, but not including, what is necessary to monitorcomputer and user activity When its “footprint” (i.e. amount ofresources used when not receiving a package) is low, the packagereceiver 56 could silently discard all such announcements

[0145] When package reception has either base or all package receptionenabled, the package receiver 56 evaluates each multicast announcementpacket to determine whether the corresponding package should bereceived. Reception should commence only if: (1) reception of the typeof package (base or delta) is currently enabled; (2) the package has notalready been successfully received; (3) the package is for a subscribedchannel; and (4) disk/memory usage management permits the reception ofthe package (explained in detail below).

[0146] If the package receiver 56 determines that a package should bereceived, the package receiver 56 requests the multicast receiver 54 toenable the associated address(es). The package receiver 56 thenprocesses the package's packets, discarding packets already received andstoring previously unreceived packets in memory (i.e. writing them todisk), thus reassembling the package.

[0147] Preferably, the package receiver 56 monitors the packet sequencenumbers and first and last packet indications to determine whether theentire package has been received without lost packets. The packagereceiver 56 identifies the end of a package's transmission by either notreceiving any packets within a predetermined timeout period or byreceipt of the last packet In multicast networks that allowout-of-sequence packets, the package receiver may have to wait foranother timeout period to determine that all packets have been received.If the package was received intact, the package receiver 56 makes thepackage available to the content viewer 58. If the package failed to bereceived intact, the package receiver may discard the package,preferably without interruption of or notification to other operations.Preferably, however, the set of missing packets is recorded (in somefashion) and on a repeat transmission of the packets, the “holes” arefilled in by storing only missing packets. This provides a higherprobability of reception on repeat transmission, but may not benecessary with some networks having a very low packet loss probability.

[0148] Preferably, while the package is being received, the packagereceiver 56 also optionally offers the user the opportunity to abort thereception if, for example, the receiver 56 is needed for otheractivities. While the package is being received, the package receiver 56also preferably monitors user activity and computer terminal loading andaborts reception depending on previously configured user preferences.

[0149] Memory/Disk Space Management

[0150] Prior multicast systems often interfered with the normaloperation of the receiver by consuming too much memory or disk space.Because the present invention's high-speed multicast network permits thetransmission of larger amounts of data than low speed multicasts, thereis a high probability that, without effective memory/disk spacemanagement, the packages would consume too much memory/disk space. Inorder to overcome this potential problem, the web crawler 30 and packagereceiver 56 of the present invention cooperate to manage memory usage tominimize impact on the receiver 26.

[0151] First, each channel definition 32 that is provided to the webcrawler 30 includes a “memory budget” for the channel. (FIG. 3). Thememory budget, which approximates the size of the channel, is used toensure that WebCast's memory/disk usage does not exceed userexpectations. The web crawler 30 ensures that the total memory spaceconsumed by a channel does not exceed the channel's memory budget.Specifically, the web crawler 30 checks the size of each package priorto submitting it to the package delivery subsystem 36 for transmission.The web crawler 30 only submits a base package to the package deliverysubsystem 36 when it does not exceed the memory budget. The web crawler30 only submits a delta package when the delta package size plus thecorresponding base package size do not exceed the memory budget.

[0152] The package receiver 56 also checks the size of a package priorto beginning its reception. The package receiver preliminarily permitsreception when the size of the package plus the size of any previouslyreceived packages does not exceed the channel's memory budget. If thepackage size exceeds the memory budget, the package receiver checkswhether deleting the channel's previously received packages will allowreception. For a base package, the package receiver deletes thechannel's packages as necessary to make room for the base package.Preferably, the package receiver 56 first deletes the previous basepackage, then any delta packages for that previous base package and thenany other delta packages as necessary. For a delta package, the packagereceiver preferably deletes the channel's previously received deltapackages as necessary to permit reception.

[0153] Once preliminary reception of a package is permitted, the packagereceiver 56 checks the available space in the receiver memory 28 againsta predetermined minimum memory space that is not to be used for WebCastchannel storage. The package receiver only permits reception when thepackage will not take available memory/disk space under this threshold.

[0154] Preferably, the package receiver records when memory/disk spacedoes not permit a package's reception and the content viewer 58 notifiesthe user of the shortage of memory space. In a preferred embodiment, theuser is notified through the EPG (Electronic Program Guide). Prior tosubscribing to a package, the EPG notifies the user of the memory budgetfor the channel and the available memory/disk space. FIG. 6 is anexample of an EPG window that may be used to so notify the user. Asillustrated in FIG. 6, the EPG also preferably notifies the user whenreception of packages will take place. For example, base packages may bebroadcast during a slow use period (such as overnight). Delta packages(containing updates) may preferably be broadcast on a periodic basis,such as every half hour. The EPG may also allow a user to specify whenpackages can be received.

[0155] Channel Subscription and Conditional Access

[0156] As discussed above, every user receives an Electronic ProgramGuide (EPG) channel In the preferred embodiment, there is a single EPGchannel. However, multiple EPG channels may be used which arespecifically tailored to a user based on factors such as language, theservice plan a user has selected, age, sex, etc. The EPG preferablycontains promotional content of each available channel which allows auser to evaluate the channels and select which channels to subscribe toor unsubscribe from. Preferably, the EPG and its promotional content isstructured like a web site—it is a collection of HTML pages withembedded graphics and other active content and a user indicates thedesired channels by clicking on buttons, check boxes, etc. FIG. 7 is anexample of an EPG window that may be used to notify a user of theavailable channels. The FIG. 7 example also allows a user to link to apreview of the channels and provides a box for the user to subscribe orunsubscribe to a channel.

[0157] When launched by the user, the content viewer 58 in the receiver26 makes the promotional material available to the user. The contentviewer 58 also processes a user's requests to subscribe to orunsubscribe a channel. If no conditional access is implemented, thecontent viewer 58 informs the package receiver 56 of the user'ssubscription/unsubscription request and the package receiver 56 startsreceiving or ceases reception of the channel's packages as appropriate.

[0158] Preferably, however, a conditional access scheme is implementedin the present invention. As discussed previously, the conditionalaccess may be implemented, for example, by the multicast network 24, bythe back-end subsystem 22, or by a combination of the two.

[0159] If the conditional access is implemented by the conditionalaccess system 25 in the multicast network 24, the content viewer 58performs a subscribe or unsubscribe transaction against the multicastnetwork. This is typically performed via the multicast receiver 54,which may optionally contact the head-end subsystem 52 eitherimmediately or at a later time. For some multicast networks, the contentviewer 58 may directly contact the head-end subsystem 52 (via theInternet, dialup modem connection, or any other suitable means) andperform the transaction. If the transaction is accepted, the contentviewer 58 informs the package receiver 58 which, in turn, startsreceiving or ceases reception of the channel's packages as appropriate.Preferably, the multicast network 24 provides the complete set ofcurrently subscribed channels at the end of such a transaction (or uponrequest), thereby keeping the content viewer 58 and multicast receiver54 synchronized with the multicast network 24. Also, the conditionalaccess system 25 preferably uses encryption to prevent unauthorizedaccess to a multicast address, wherein the keys used to encrypt onechannel's multicast addresses are different from the keys used toencrypt other channel's multicast addresses.

[0160] Alternatively, the multicast network 24 implements theconditional access and the back end subsystem 22 performs thesubscription processing. With this arrangement, the back end subsystem22 processes transactions for channel subscriptions and unsubscriptionsand the multicast network 24 controls access to multicast addresses. Thecontent viewer 58 contacts the registration server 46 of the back endsubsystem 22 (via dial-up modem, the Internet, etc.) and performs asubscribe transaction The transaction request includes information: (i)identifying the multicast receiver that the receiver is receiving from,(ii) the channel being requested, and (iii) optionally, information usedto authenticate the sender of the request.

[0161] The registration server 46 authenticates the request and, if themessage is authenticated, maps the channel requested to the set ofmulticast addresses carrying the channel. The registration server 46then performs a transaction against the multicast network head-endsubsystem 52 requesting that the receiver 26 be given access to theaddresses that carry the subscribed-to channel. The authentication isintended to verify that the request came from the content viewer 58 ofthe receiver 26 for which access to a channel is being requested.Preferably, the authentication is performed via password authentication.However, authentication may also be performed using public keyencryption

[0162] Once the subscription requests have been processed, the multicastnetwork 24 is responsible for ensuring that the receivers 26 onlyreceive packets from authorized multicast addresses. In the preferredembodiment, the conditional access system 25 in the multicast networkutilizes encryption to prevent unauthorized access to a multicastaddress wherein the keys used to encrypt one channel's multicastaddresses are different from the keys used to encrypt other channel'smulticast addresses.

[0163] Alternatively, conditional access may be implemented solely bythe back end subsystem 22. In this embodiment, the content viewer 58contacts the registration server 46 (via dial-up modem, the Internet,etc) and performs a subscribe/unsubscribe transaction. The back-endsubsystem 22 then makes the channel's decryption keys available to thereceiver The receiver may have to periodically perform transactions withthe registration server to obtain and update encryption keys. (Thisoptionally may be performed along with usage reporting.) If thetransaction is accepted, the package receiver 56 is informed, asdescribed above This conditional access embodiment is typically not assecure as the conditional access system offered by the multicastnetwork, but still provides some level of security.

[0164] In any of the above-described embodiments, multiple subscriptionand/or unsubscription requests may be batched together for moreefficient processing

[0165] Offline Browsing

[0166] Offline browsing refers to a user accessing web site contentwithout connecting to the Internet. The content viewer 58 within thereceiver 26 is responsible for supporting the user's offline browsing ofWebCast channel content In a preferred embodiment, the content viewer 58comprises a proxy-server application which works with an existing (i e.unmodified) web browser 12 This allows the user to continue to work witha popular web browser without relearning a new user interface.

[0167] In the preferred embodiment, when the user initiates an offlinebrowsing session, the content viewer 58 first determines whether the webbrowser 12 is running and, if so, prompts the user to close the browser.If the browser is not running, the content viewer 58 configures thebrowser to access the Internet via the content viewer/proxy-server. Thecontent viewer/proxy-server then launches the browser to display theElectronic Program Guide (EPG). At the end of an offline browsingsession, the content viewer/proxy-server 58 closes the browser 12 andreconfigures the browser to its original, unproxied configuration.

[0168] In another embodiment of the invention, the content viewer 58 isintegrated with a browser's cache so that the browser requests URLs fromthe content viewer 58 prior to requesting them across the network.

[0169] In either embodiment, once the content viewer 58 has initialized,it receives requests for URLs from the browser 12 and attempts to findthem from its “cache” of URLs. The cache consists of the URL data itemscontained within the received packages from subscribed channels. Fromthe standpoint of the content viewer 58 performing a cache lookup, a URLconsists of two parts. For example, the URL “http.//wwwdirecpc.com/users/index.html” illustrates these two parts

[0170] (1) “http://www.direcpc.com” —the first part identifies theprotocol (which is always http) and the domain name of the web serverfrom which the URL is being requested,

[0171] (2) “/users/index html” —the second part identifies the pathname(directory and filename) of the URL within the web server's file system.

[0172] The content viewer 58 performs a cache lookup by first performinga domain name lookup. The content viewer 58 maintains a data structurethat maps a domain name to the list of channels which contain URLs fromthe domain name. In a preferred embodiment, this data structure is ahash table, wherein each hash entry is a list and each list entrycontains a domain name and a channel ID. (As is known in the art, a hashtable provides an efficient method of information retrieval from anarray Use of a hash table allows many of the different possible entries(or keys) that might occur to be mapped to the same location in thearray under the action of an index function.) Preferably, the contentviewer 58 constructs this data structure from the supplementalinformation within each package listing the domain names of URLsincluded in the package. The content viewer 58 extracts the domain namefrom the URL and determines from this data structure the list ofchannels which might contain the URL.

[0173] The content viewer 58 next performs a URL lookup, wherein thecontent viewer takes the list of candidate channels (determined from thedomain name lookup) and check's each channel's packages until a match isfound. Preferably, the content viewer 58 first checks a channel's deltapackage (if available) for a match and then its base package

[0174] In some instances, a ILL may exist in more than one channel'spackages. To handle this, in a preferred embodiment, the content viewer58 checks channels starting with the most recent crawl start time. (Achannel's start time is recorded in its package's supplementalinformation.)

[0175] Upon finding the URL in a package, the content viewer 58retrieves the URL data item from the package from memory 28 anddecompresses the URL data item, if necessary If the URL data item isdifference compressed, the content viewer retrieves the URL data itemfrom both the base and delta packages (decompressing as necessary) andperforms difference decompression to restore the revised URL's content

[0176] In many cases, the present invention is used with a receiver 26having a switched connection to the Internet, such as a home computerrunning Windows95® with a dialup modem telephone connection to theInternet. The telephone line is often used both for Internet access andfor ordinary telephony services (POTS). Another example is an ISDNswitched connection. In addition to being a shared resource, where thereceiver may interfere with other use of the connection, a switchedconnection to the Internet is frequently metered so that a user incursan additional expense with each use.

[0177] In such switched connection situations, it is desirable that thereceiver share the telephone line with other users. In a preferredembodiment, the content viewer 58 interacts with the user to allow“seamless” access to content outside of the channel definition whilecontrolling access to the phone line in a way which permits sharing thephone line with other users. The content viewer does this by controlling“cache misses.”

[0178] A “cache miss” occurs when the browser 12 requests a URL which isnot within any of the packages that have been received and stored inmemory 28. The requested URL may not be present in a received packagebecause, for example, a cached web page may reference URLs which werenot crawled and incorporated into a channel. This may occur either dueto Internet outages during the crawling or because the web pagecontained dynamic content which could not be automatically crawled. Inthis instance, there are often several cache misses within a shortperiod of time, one for each missing embedded URL.

[0179] The requested URL may also not be present if the user clicked ona link from a page within the channel definition which references a pageoutside of the channel definition. For example, if the channeldefinition defined a search depth of two levels, and the user requesteda URL that was three levels deep, a cache miss would occur. A cache missmay also occur if the user directed the browser to a URL which is notpresent in the cache.

[0180] Preferably, when a cache miss occurs, the content viewer 58notifies the user and allows the opportunity for the user to connect tothe Internet (via the switched connection) to obtain seamless access touncached web pages. FIG. 8 is a flowchart of the steps performed by thecontent viewer 58 when a cache miss occurs. The content viewer 58 firstdetermines whether another cache miss has recently occurred (block 100).If not, it then determines whether the user has previously specified howa cache miss should be treated (block 102). If no user preferences arespecified, the content viewer 58 notifies the user that a cache miss hasoccurred and that the requested content can only be obtained byaccessing the Internet (block 104).

[0181] The content viewer 58 then queries the user whether they want toinitiate a connection to the Internet to retrieve the requested content(block 106). This query prevents an automatic connection to the Internetthat may interfere with other uses of the connection (i.e. interrupt atelephone call) or may incur undesired charges. If other cache missoccurs while waiting for a response to the query, the content viewer 58holds the cache misses until the user responds. The block 106 alsooptionally initiates a countdown along with the query that lets the userknow that they have N seconds to respond to the query. FIG. 9 is anexample of a dialog box that may be used to notify the user of the cachemiss and queries the user about connecting to the Internet. The exampleof FIG. 9 also includes the optional countdown

[0182] Referring back to FIG. 8, if the user responds that they do notwish to connect to the Internet or if the countdown expires with no userresponse, the content viewer 58 denies the cache miss request byproviding an indication to the browser 12 that the content can only beobtained by connecting to the Internet (block 108). In a preferredembodiment, the content viewer returns an “error” HTTP responsecontaining a text or HTML body (or a redirection to an HTML page) whichcontains text explaining that the content can only be obtained via theInternet. This results in the browser 12 displaying this text (or HTMLpage) to the user.

[0183] Alternatively, if the user responds that they wish to access therequested content by connecting to the Internet, the content viewerinitiates the Internet connection (block 110). In a preferredembodiment, the content viewer 58 acts as a normal proxy-server andinitiates TCP connections to the requested URL's server and to theservers of other requested URLs from cache misses which were held whilewaiting for the user response. Upon receiving such a connectioninitiation, the receiver 26 may then be configured to automaticallyinitiate a switched connection to the Internet. In other embodiments,the content viewer 58 may interact with the switched connection's devicedriver to initiate a connection and hold all URL requests until afterthe connection to the Internet has been established.

[0184] Referring back to the block 100, if the content viewer 58determines that another cache miss has recently occurred, it follows theinput given by the user for the previous cache miss and either initiatesan Internet connection (block 110) or denies the request (block 108).The blocks 110-112 prevent the user from being “peppered” with cachemiss queries, which may occur frequently in web pages with more than onemissing embedded URL.

[0185] Similarly, if the content viewer determines that the user hasindicated a preference for cache miss requests (block 102), it followsthat preference (block 114) and either initiates the connection (block110) or denies the request (block 108).

[0186] Usage Reporting

[0187] The present invention provides sufficient usage information to aweb site to support an advertising-based business model while requiringno change to a web site's operation. Alternatively, usage informationmay be reported to another source, such as a network operations center(NOC). Specifically, the present invention may report usage informationto the web site as if the user were accessing the web site via aconventional caching proxy-server. Alternatively, the present inventionmay report usage information to the web site in the form of log files.

[0188] Generally, a web browser 12 sends to a web server 10 a separaterequest for each URL which a user views. (FIG. 1). A request for a URLis called a “hit”. To generate advertising revenue, a web site typicallyrecords for each advertisement URL the number of “hits” (i.e., thenumber of times the site has served the URL). the web site receivesadvertising revenue based on either: (i) the actual number of “hits” thesite receives, or (ii) a forecast of the number of “hits” thatadvertisement can be expected to receive based on past experience.

[0189] A web site is typically unable to determine the identity (i.e.,name, address, phone number, etc.) of the users visiting the site.However, a web site is able to tell which URLs are being accessed overtime (i.e., weeks and months) by the same user. The web site collectsthis information by assigning each different browser requesting URLsfrom the site a unique “cookie”. (As is known in the art, a “cookie” isa piece of limited, internal information transmitted between a webserver and a web browser which uniquely identifies the user withoutrevealing the identity of the user.) It is important, in order tomaintain advertising revenue, for a web site to be able to tell which“hits” are being received from the same user. The present inventionprovides such information to the web sites through use of the cache hittracker 40 in the back-end subsystem 22. (See FIG. 3.)

[0190] Referring also to FIG. 10, the content viewer 58 in the receiver26 sends usage information 42 to the back-end subsystem's cache hittracker(s) 40 as specified by the various channel's channel definitions.The hit tracking part of the channel definition is delivered to thereceiver 26 within a package's supplemental information. In a preferredembodiment, the usage information consists of a separate record for each“hit,” wherein only a subset of the cache hits are reported. This usageinformation is preferably reported via a TCP/IP network, such as theInternet 14, and the receiver's interface to the Internet may be via aswitched connection such as dialup modem connection. It is understood,however, that the usage information could be reported via any othersuitable connection.

[0191] The present invention returns usage information to a web site (orother location) on a per channel basis either by proxy (which requiresno change to web site operation) or by log files (which requires nochange to web site operation other than simply acquiring and processingthe log files). With proxy reporting, the cache hit tracker 40preferably operates (from the web servers' perspective) as an HTTP proxyserver. The cache hit tracker 40 stores reported “hits” from the contentviewer 58 in the receiver 26 and subsequently performs a HTTP operationfor each reported URL. This minimizes the time that the receiver 26needs to be connected to the Internet 14 as it is not delayed while itshits are reported to the web servers 10. The type of HTTP operationperformed by the cache hit tracker 40 is defined by the channeldefinitions, which are sent along with the usage reports 42 to the cachehit tracker 40. Specifically, the following HTTP operations may beconfigured to perform proxy usage reporting on a per channel basis:

[0192] (1) GET—The cache hit tracker 40 may perform an HTTP GEToperation (which, generally, transfers URL data item from the web server10 to the cache hit tracker 40.) The cache hit tracker 40 then waits forall the data to be received, but discards the data as it is received.This operation most closely matches what the web server 10 would havereceived had the user directly retrieved the URL data item without useof the present invention.

[0193] (2) GET no wait—The cache hit tracker 40 may perform an HTTP GEToperation Oust as above) but waits only for the HTTP response header tobe received from the web server 10. Once received, the cache hit tracker40 immediately closes the connection. This operation uses less networkbandwidth, as the actual URL data item need not be completelytransferred.

[0194] (3) GET If Modified Since—The cache hit tracker 40 may perform aGET If-Modified-Since HTTP operation, wherein the last-modified datacomes from the HTTP response header included with the URL in thepackage. This operation also reduces network utilization, as the fullcontent would not typically be sent, while completing the HTTPoperation.

[0195] (4) GET If Modified Since No Wait—The cache hit tracker 40 mayperform a GET If-Modified-Since operation (as above), but does not waitfor any data after the response from the web server 10 that the URL haschanged and should be sent. This operation uses less network bandwidth,as the actual URL data item is not transferred.

[0196] The cache hit tracker 40 may also perform usage reporting usinglog files 44. In this method, the cache hit tracker 40 stores reportedhits in log files 44, preferably maintaining separate log files for eachchannel. The cache hit tracker 40 either makes the log files 44available to the web site (via, for example, an FTP GET operation) orperiodically delivers the log files 44 to the web site (via, forexample, an FTP PUT operation or e-mail).

[0197] In a preferred embodiment, each hit log file 44 is a flat ASCIIfile, with one record per line and a fixed number of fields per record.All fields are space separated and fields which may contain spaces areenclosed in quotes(“”). TABLE 2 Field Name Field Format DescriptionChannelID Four (4) alphanumerics Identifies the channel this hit camefrom URL “printable ASCII” The URL from the HTTP (enclosed in quotes)request's Request Line UserAgent “printable ASCII” From the HTTPrequest's User- (enclosed in quotes) Agent line. If the User-Agent lineis not present, the field is “NOAGENT” HitTime YYYYMMHHMMSS GMTtimestamp of when the user accessed the requested URL data item from thecache ReportDelay Eight (8) numerics Duration (in seconds) from when thecache hit occurred to when the hit was reported (either to theoriginating server or the file) EditionID RRR MIMM An ID which uniquelyidentifies the snapshot of the channel's content given to the user. (RRRis the three least significant digits (LSD) of the base package edition# and MMM is the three LSDs of the delta package edition #.) SiteiD Ten(10) alphanumerics Uniquely, but anonymously, identifies the userHitMethod Four (4) Identifies how the hit should be alphanumericsrelayed to the web server. Will contain one of FILE - sent in log fileGETN - complete HTTP GET operation GETR - HTTP GET No Wait (connectionclosed after response header is received) GETI - HTTP Get If ModifiedSince (default) GITR - HTTP Get If Modified Since No Wait (connectionclosed after response header is received)

[0198] In the preferred hit log file format of Table 2, the “SiteID”field provides the web site with a rough equivalent to a “cookie”(discussed above) in that it allows hits from a single user to beidentified, without revealing the identity of that user. The followingis an example of a hit log file record (in an actual log file the recordoccupies a single line)

[0199] EONL “http:/www hns.com/index2.html” “Mozilla/2 0 (compatible,MSIE 3.0; Windows 95)” 19770723164419 00002000 146.001 09AKLM5823 FILE

[0200] The channel definition for each channel dictates whether all hitsor a filtered subset of hits should be reported. Filtering the reportedhits minimizes unnecessary processing in the web server (and connecttime for the user), while retaining reporting information foradvertising and other critical hits. Filters are typically establishedto filter out “insignificant hits” and report only advertising-relatedhits and HTML hits, thereby allowing a web site to tally advertisingstatistics and page popularity. The filters that may be used to reducethe reporting of insignificant hits include:

[0201] (1) Content Filters—Content filters may be used, for example, ifall advertisements are in the form of .gif (graphics interchange format)images. A content filter would detect and report all gif image URLs andwould filter out most non-.gif (i.e. non-advertisement) URLs.

[0202] (2) URL Filters—URL filters may be used, for example, if alladvertisements have URLs that contain “/ads/” in their URL. Thus, a “URLcontains ‘/ads/’” filter would detect and report all hits with a URLcontaining “lads/’” and would filter out other (i.e. non-advertisement)URLs. Variations of such a URL filter may include a “URL StartsWith”filter, a “URL Ends With” filter, or a “URL matches a regularexpression” filter, etc

[0203] The channel definition also defines under what circumstances thereceiver should report usage information for that channel. For example,the receiver may report usage information for a channel at the end of anoffline browsing session or on a periodic basis (overnight, weekly,monthly, etc ) These two methods are referred to as “session endreporting” and “periodic reporting,” respectively. Alternatively, usageinformation may be reported on demand or at any other times.

[0204] A WebCast channel generally contains just a subset of a typicalweb site's content. As discussed above in connection with FIG. 8, when auser requests access an uncached URL (a cache miss), the content viewer58, at the user's discretion, may connect the receiver 26 to theInternet and access the URL directly from the web server 10. This directconnection allows the web server 10 to track such hits as it normallywould with any Internet user. Thus, the cache miss feature of thepresent invention further allows a web site to retain its tracking of“click-thrus”, i e , cases where a user clicks on to link to a web sitewhich provides additional information. Advertisements contained within aweb site, for example, often contain links to a web site providingadditional information on the advertised products This is importantbecause advertising revenue is sometimes based not only on the number ofusers to view an advertisement, but also on the number of users that“click-thru” for further information on the advertised product. Byproviding “seamless” or automatic access to URLs outside of the channeldefinition, the present invention preserves this important form ofadvertising revenue. However, in such a situation it is desirable thatusage reporting does not interfere with other uses of the switchedconnection and does not incur excessive connection charges.

[0205] Utilizing both session end and periodic reporting helps meetthese goals. Channels are configured for session end usage reportingwhen getting usage reports in close to real-time is more important thanminimizing the number of usage reporting connections Channels areconfigured for periodic reporting when minimizing connection charges isimportant. Usage reporting can be concentrated into one connection perweek by configuring all channels for weekly reporting The usagereporting for all channels can also be scheduled for the same range oftime

[0206] With periodic reporting, usage reporting is optimized to minimizethe number of connections to the Internet. This is done by waiting aslong as possible to send in usage information and to “piggyback” usagereports to any other connection to the Internet which might take placein the meantime. (“Piggyback” refers to sending the usage informationalong with other information.)

[0207] With periodic reporting, it is desirable to balance the reportingof content so that the cache hit trackers are not overloaded. A receiverreceives a usage reporting time range (e.g, every night between midnightand 5 am) and, for each channel, a usage connection timeout value,indicating the maximum time that usage information should wait prior tobeing reported. This timeout may have, for example, a default of fivedays. The receiver picks a random time within the usage reporting timerange and, at that time, checks whether a usage reporting connectionshould be made. A connection should be made if any channel's usageinformation is older than the channel's usage connection timeout. Whensuch a connection is made, all pending usage information is reported.

[0208] In a preferred embodiment, the receiver “piggybacks” usagereporting on other Internet connections by first receiving a piggybacktimeout and a usage reporting retry timeout. The receiver also maintainsa usage reporting time which is either disabled, running or expired. Thereceiver monitors the Internet connection. When the receiver connects tothe Internet, the receiver initializes the usage reporting timer totimeout after the piggyback timeout and starts the timer. When thereceiver disconnects from the Internet, it disables the usage reportingtimer. When the usage reporting timer expires, the receiver checks ifthere is any usage information ready to be reported and, if there is,the receiver attempts to report it. When the report either succeeds orfails, the receiver checks the connection, and if still connected, thereceiver initializes the usage reporting timer to timeout after theretry timeout period and starts the timer.

[0209] Thus, with periodic reporting, an extra connection to theInternet for usage reporting only occurs if the user connects to theInternet less than once during the usage connection timeout value (e.g.5 days).

[0210] With session end usage reporting, the reporting normally takesplace at the end of an offline browsing session. If the user indicatesat that time that usage should not be reported, (because, for example,the phone line is already occupied), the usage report is stored untilthe start of the next offline browsing session. Thus, at the start ofeach offline browsing session, the content viewer 58 checks if there isa session end usage report pending from a previous offline browsingsession. If so, the content viewer 58 prompts the user for permission toreport the usage information and only allows offline browsing tocommence after permission is received and the usage has been reported.Once session end usage has been reported, the content viewer 58 willimmediately thereafter report any pending periodic usage, as this mayeliminate the need for a separate periodic usage report connection.

[0211]FIG. 11 is a flowchart illustrating the steps performed by thecontent viewer 58 in the preferred embodiment for either session end orperiodic usage reporting. Once the content viewer has determined thatusage reporting should be initiated, it determines whether the receiver26 is already connected to the Internet (block 150). If not, it promptsthe user for permission to connect to the Internet to facilitate usagereporting (block 152). This prevents usage reporting from interferingwith other receiver applications. Preferably, the content viewer 58 alsoinitiates a countdown for the user to respond FIG. 12 is an example of adialog box used to prompt the user for permission to connect to theInternet for usage reporting. The FIG. 12 example also includes theoptional countdown.

[0212] Referring back to FIG. 11, the content viewer then determineswhether the user granted permission to connect (block 154). Ifpermission was granted (or if the user failed to respond within thespecified time period), the content viewer 58 attempts to establish aconnection to the Internet (block 156).

[0213] Next, the content viewer determines whether the connection wassuccessful (block 158). If the attempt to connect failed, or if the userdenied permission to connect, the content viewer stores the usageinformation for reporting at a later time (block 160) and the programterminates. If the connection was successful the content viewer reportsall pending usage information to the cache hit tracker 40 (block 162).The content viewer preferably reports the usage information in the formof a sequence of HTTP PUT or HTTP POST operations. These HTTP operationsare preferable because they are almost always allowed to pass throughany firewalls onto the Internet. However, the usage information couldalso be reported via e-mail, which allows information to be transmittedin nearly any kind of system. Preferably, each HTTP PUT or POSToperation (or e-mail transmission) contains a block of usage informationfor multiple hits, which allows for more efficient transmission of usageinformation by reducing usage reporting connection time.

[0214] The content viewer then determines whether the report wassuccessful (block 164). If not (for example, if the HTTP PUT operationfails), it stores the usage information to be reported at a later time(block 166) It may also determine whether there have been severalreporting attempts and, if so, may discard the usage information (block166) After the usage information has been stored or discarded (block166), or if the content viewer determines that the report was successful(block 164), the content viewer disconnects from the Internet (block168).

[0215] Alternatively, if the content viewer 58 determines that thereceiver is connected to the Internet (block 150), it “piggybacks” allpending usage information onto the current Internet connection (block170), as discussed above. The content viewer then determines whether thereport was successful (block 172). If not, the content viewer eitherstores or discards the usage information (block 174), as described abovein connection with block 166. If the report was successful or after theusage information is stored or discarded, the program terminates.

[0216] Numerous modifications and alternative embodiments of theinvention will be apparent to those skilled in the art in view of theforegoing description. Accordingly, this description is to be construedas illustrative only and is for the purpose of teaching those skilled inthe art the best mode of carrying out the invention The details of thesystem may be varied substantially without departing from the spirit ofthe invention, and the exclusive use of all modifications which arewithin the scope of the appended claims is reserved.

1. An apparatus that transmits content organized into channels, whereina channel's content includes a plurality of URL data items and each URLdata item is addressed by a URL, the system comprising: means forassigning one or more multicast addresses to each channel; means forscheduling the assembling of a channel's content; means for assemblingthe channel's content, means for fragmenting the channel's content intopackets, wherein each packet is addressed with one of the channel'smulticast addresses; and means for multicasting the packets. 2 Theapparatus of claim 1, wherein the means for multicasting the packetsincludes means for transmitting the packets to a multicast receiver of amulticast network.
 3. The apparatus of claim 1, further comprising meansfor encrypting a subset of a channel's packets prior to multicasting,wherein the encryption means encrypts either all or a part of the packetand wherein each channel's packets are encrypted with a set ofencryption keys which are unique to that channel 4 The apparatus ofclaim 3, further comprising. means for receiving requests from areceiver of the multicast for access to a channel's packets, means formapping the requested channel to the multicast addresses that carry thechannel's packets, and means for requesting authorization for thereceiver to access the requested channel's packets.
 5. The apparatus ofclaim 4, further comprising means for authenticating the requests toensure that the requests originated from the receiver for which accessis being requested. 6 The apparatus of claim 2, wherein the multicastnetwork is a geosynchronous satellite digital TV broadcast system. 7 Theapparatus of claim 1, wherein the multicast network is a one-way cableTV network. 8 The apparatus of claim 1, wherein the multicast network isa digital video broadcast (DVB) network.
 9. The apparatus of claim 1,wherein the packets are multicast to a plurality of receivers
 10. Theapparatus of claim 9, wherein a channel's content includes indexinginformation which allows URL data items contained within the channel'scontent to be quickly looked up by the receiver which receives thechannel's content.
 11. The apparatus of claim 10, wherein the channel'scontent further includes a data structure containing each domain namepresent in the URLs of the URL data items within the channel's content.12 The apparatus of claim 9, further comprising a conditional accesssystem for controlling each receiver's access to packets, wherein eachreceiver can only access packets which contain multicast addresses whichthe conditional access system has authorized the receiver to access 13.The apparatus of claim 12, wherein the means for multicasting thepackets is a geosynchronous satellite digital TV broadcast earthstation.
 14. The apparatus of claim 12, further comprising: means forreceiving requests from the receivers to obtain access to a channel'spackets, means for mapping the requested channel to the multicastaddresses that carry the channel's packets, and means for authorizingthe receivers' access to a channel's packets in response to thereceivers' request for access.
 15. The apparatus of claim 13, wherein achannel's content includes indexing information which allows URL dataitems contained within the channel's content to be quickly looked up bythe receiver which receives the channel's content, the system furthercomprising. means for scheduling a configurable number ofretransmissions of the channel's previously assembled content; means forfragmenting and multicasting the channel's content according to theschedule; and means for specifying the transmission rate of thechannel's content, wherein the packets containing the channel's contentare multicast at the specified rate.
 16. The apparatus of claim 13,further comprising means for compressing a subset of the URL data items,wherein each URL data item is compressed individually independent ofother URL data items such that each compressed URL data item can bedecompressed without decompressing other URL data items.
 17. Theapparatus of claim 16, wherein the URL data items are compressed with alossless data compression algorithm
 18. The apparatus of claim 1,further comprising: means for scheduling a configurable number ofretransmissions of a channel's previously assembled content, and meansfor fragmenting and multicasting the channel's content according to theschedule.
 19. The apparatus of claim 18, further comprising means forspecifying a transmission rate of a channel's content, wherein thepackets containing the channel's content are multicast at the specifiedrate.
 20. The apparatus of claim 19, further comprising: means forassigning one or more multicast addresses to an announcement packet,wherein the announcement packet includes an announcement of an upcomingtransmission of a channel's content; and means for multicasting theannouncement packet prior to the multicast of the packets containing thechannel's content.
 21. The apparatus of claim 19, wherein the channel'scontent includes a data structure containing each domain name present inthe URLs of the URL data items within the channel's content.
 22. Theapparatus of claim 19, wherein the packets are multicast to a pluralityof receivers and wherein a channel's content includes indexinginformation which allows URL data items contained within the channel'scontent to be quickly looked up by the receiver which receives thechannel's content.
 23. The apparatus of claim 22, wherein the channel'scontent further includes a data structure containing each domain namepresent in the URLs of the URL data items within the channel's content.24. The apparatus of claim 1, wherein a channel's content includes adata structure containing each domain name present in the URLs of theURL data items within the channel's content.
 25. The apparatus of claim1, wherein the means for assembling the channel's content furthercomprises: means for assembling a base package of the channel's content,wherein the base package contains each URL data item in the channel; andmeans for assembling a delta package of the channel's content, whereinthe delta package contains URL data items which have changed or are newsince the previous assembling of the base package
 26. An apparatus thattransmits content organized into channels, wherein a channel's contentincludes a plurality of URL data items and each URL data item isaddressed by a URL, the apparatus comprising means for scheduling theassembling of a channel's content; means for assembling the channel'scontent, means for compressing a subset of the URL data items, whereineach URL data item is compressed individually independent of other URLdata items such that each compressed URL data item can be decompressedwithout decompressing other URL data items; means for fragmenting thechannel's content into packets, and means for multicasting the packets27. The apparatus of claim 26, wherein the URL data items are compressedwith a lossless data compression algorithm
 28. The apparatus of claim26, wherein the means for assembling the channel's content furthercomprises: means for assembling a base package of the channel's content,wherein the base package contains each URL data item in the channel; andmeans for assembling a delta package of the channel's content, whereinthe delta package contains URL data items which have changed or are newsince the previous assembling of the base package 29 The apparatus ofclaim 28, wherein the means for scheduling the assembling of thechannel's content comprises means for scheduling the assembling of thebase package and means for scheduling the assembling of the deltapackage.
 30. The apparatus of claim 28, further comprising means fordifference compressing a subset of the URL data items in a channel'scontent which is present in both the delta package and the previous basepackage. 31 The apparatus of claim 30, wherein the differencecompression means further comprises means for dividing a URL data itemin the delta package into sections, and for each section, means forplacing into a compressed version of the URL data item, one of areference to where that section can be found in the base package, or thesection of URL data item from the delta package.
 32. The apparatus ofclaim 28, further comprising means for assembling a second delta packagewhich contains a subject of the URL data items which have changed or arenew since the assembling of the previous delta package.
 33. Theapparatus of claim 26, further comprising means for encrypting a subsetof a channel's packets prior to transmission, wherein the encryptionmeans encrypts either all or part of the packet and wherein eachchannel's packets are encrypted with a set of encryption keys which areunique to that channel.
 34. An apparatus that transmits contentorganized into channels, wherein a channel's content includes aplurality of URL data items and each URL data item is addressed by aURL, the apparatus comprising: means for assembling a base package of achannel's content, wherein the base package contains each URL data itemin the channel; means for fragmenting the base package into packets;means for multicasting the base package packets to a plurality ofreceivers, means for assembling a delta package of a channel's content,wherein the delta package contains URL data items which have changed orare new since the previous assembling of the base package; means forfragmenting the delta package into packets; and means for multicastingthe delta package packets to the plurality of receivers.
 35. Theapparatus of claim 34, wherein some of the receivers comprise a personalcomputer.
 36. The system of claim 34, wherein some of the receiverscomprise a set top box. 37 The apparatus of claim 34, further comprisingmeans for scheduling the assembling of base packages and delta packages,wherein the base packages and delta packages are assembled according tothe schedule.
 38. The apparatus of claim 34, further comprising meansfor scheduling the multicast transmission of base package packets andfor scheduling subsequent periodic multicast transmission of deltapackage packets, wherein the base package packets and delta packagepackets are multicast according to the schedule
 39. The apparatus ofclaim 38, wherein base package packets are scheduled for transmission ata time when the receiver is not likely to be in use for otherapplications.
 40. The apparatus of claim 39, wherein the base packagepackets are scheduled for transmission late at night or early in themorning
 41. The apparatus of claim 34, further comprising means forcompressing a subset of the URL data items in the base and deltapackages, wherein each URL data item is compressed individuallyindependent of other URL data items such that each compressed URL dataitem can be decompressed without decompressing other URL data items. 42.The apparatus of claim 41, wherein the URL data items are compressedwith a lossless data compression algorithm.
 43. The apparatus of claim41, further comprising means for difference compressing a subset of theURL data items that are present in both in the delta package and theprevious base package.
 44. The apparatus of claim 43, wherein thedifference compression means further comprises means for dividing a URLdata item in the delta package into sections; and for each section,means for placing into a compressed version of the URL data item, one ofa reference to where that section can be found in the base package, orthe section of URL data item from the delta package.
 45. The apparatusof claim 44, further comprising means for compressing a subset of thepreviously difference compressed URL data item with a lossless datacompression algorithm.
 46. The apparatus of claim 34, further comprisingmeans for assembling a second delta package which contains URL dataitems which have changed since the assembling of the previous deltapackage.
 47. An apparatus that transmits content organized intochannels, wherein a channel's content includes a plurality of URL dataitems and each URL data item is addressed by a URL, the apparatuscomprising: means for scheduling the assembling of a channel's content;means for assembling the channel's content according to the schedule,means for fragmenting the channel's content into packets, means formulticasting the packets to a plurality of receivers, wherein eachreceiver stores the received channel's content in a receiver memory, andmeans for receiving usage reports from each receiver, wherein each usagereport identifies a subset of URL data items from the stored URL dataitems that was accessed from the receiver memory.
 48. The apparatus ofclaim 47, further comprising means for organizing the received usagereports by channel.
 49. The apparatus of claim 47, wherein each usagereport contains information identifying a subset of URL data itemsdelivered to a web browser.
 50. The apparatus of claim 47, wherein theusage reports comprise a set of files and wherein the URL data itemsaccessed for each channel is referenced in one set of files
 51. Theapparatus of claim 47, wherein the usage reports contain informationidentifying each URL data item, from the stored URL data items, beingdelivered to a web browser.
 52. The apparatus of claim 50, wherein usagereporting is performed on a subset of a channel's URL data items and thefiles contain a separate record for each time a usage reported URL dataitem was delivered to a web browser, wherein the record identifies theURL of the URL data item.
 53. The apparatus of claim 52, wherein therecord identifies when the URL data item was delivered to the webbrowser. 54 The apparatus of claim 52, wherein the record contains afield uniquely identifying the user that accessed the URL data item 55.The apparatus of claim 54, wherein the field uniquely identifying theuser does not specific the identity of the user
 56. The apparatus ofclaim 54, wherein the field uniquely identifying the user specifies [heidentity of the user.
 57. The apparatus of claim 47, wherein a channel'scontent is assembled from a web server and further comprising means fornotifying the web server from which a URL data item was assembled thatthe URL data item was accessed by a user.
 58. The apparatus of claim 57,wherein the web server is notified that the URL data item was accessedby a user by notifying, the web server that the URL data item wasdelivered to a browser.
 59. The apparatus of claim 57, wherein the webserver is notified that the URL data item was accessed by initiating anHTTP GET operation for the URL data item
 60. The apparatus of claim 57,wherein the web server is notified of multiple accesses of multiple URLdata items by initiating an HTTP PUT operation.
 61. The apparatus ofclaim 57, wherein the web server is notified of multiple accesses ofmultiple URL data items by initiating an HTTP POST operation.
 62. Theapparatus of claim 57, wherein the web server is notified that the URLdata item was accessed by e-mail, and wherein multiple accesses ofmultiple URL data items is reported in one e-mail.
 63. The apparatus ofclaim 47, further comprising means for compressing a subset of the URLdata items; means for compressing a subset of the URL data items,wherein each URL data item is compressed individually independent ofother URL data item such that each compressed URL data item can bedecompressed without decompressing other URL data items;
 64. A methodfor multicasting content organized into channels, wherein a channel'scontent includes a plurality of URL data items and each URL data item isaddressed by a URL, the method comprising the steps of. assigning one ormulticast addresses to each channel; scheduling the assembling of eachchannel's content; assembling each channel's content according to theschedule; fragmenting each channel's content into packets, wherein eachpacket is addressed with one of the channel's multicast addresses, andtransmitting the packets via a multicast network to a plurality ofreceivers
 65. The method of claim 64, further comprising encrypting asubset of a channel's packets prior to transmitting the packets, whereineither all or a part of the packet are encrypted and wherein eachchannel's packets are encrypted with a set of encryption keys which areunique to that channel 66 The method of claim 65, further comprising thesteps of receiving requests from the receivers for access to a channel'spackets, mapping the requested channel to the multicast addresses thatcarry the channel's packets; and requesting authorization from themulticast network for the receiver to access the requested channel'spackets 67 The method of claim 66, further comprising the step ofauthenticating the requests to ensure that the requests originated fromthe receiver for which access is being requested.
 68. The method ofclaim 64, wherein a channel's content includes indexing informationwhich allows URL data items contained within the channel's content to bequickly looked up by the receiver which receives the channel's content.69. The method of claim 68, wherein the channel's content furtherincludes a data structure containing each domain name present in theURLs of the URL data items within the channel's content. 70 The methodof claim 68, wherein a channel's content includes indexing informationwhich allows URL data items contained within the channel's content to bequickly looked up by the receiver which receives the channel's content,the method further comprising the steps of. scheduling a configurablenumber of retransmissions of the channel's previously assembled content;specifying a transmission rate of the channel's content, and fragmentingand transmitting the channel's content to the receivers according to theschedule at the specified transmission rate.
 71. The method of claim 65,further comprising the step of compressing a subset of the URL dataitems, wherein each URL data item is compressed individually independentof other URL data items such that each compressed URL data item can bedecompressed without decompressing other URL data items.
 72. The methodof claim 71, wherein the URL data items are compressed with a losslessdata compression algorithm.
 73. The method of claim 64, furthercomprising the steps of. scheduling a configurable number ofretransmissions of a channel's previously assembled content, andfragmenting and transmitting the channel's content to the receiversaccording to the schedule.
 74. The method of claim 73, furthercomprising the step of specifying a transmission rate of a channel'scontent, wherein the packets containing the channel's content aretransmitted at the specified rate.
 75. The method of claim 73, furthercomprising the steps of: assigning one or more multicast addresses to anannouncement packet, wherein the announcement packet includes anannouncement of an upcoming transmission of a channel's content, andtransmitting the announcement packet to the receivers prior totransmitting the packets containing the channel's content.
 76. Themethod of claim 64, wherein the step of assembling the channel's contentfurther comprises: assembling a base package of the channel's content,wherein the base package contains each URL data item in the channel; andassembling a delta package of the channel's content, wherein the deltapackage contains URL data items which have changed or are new since theprevious assembling of the base package.
 77. A method for transmittingcontent organized into channels, wherein a channel's content includes aplurality of URL data items and each URL data item is addressed by aURL, the method comprising the steps of: scheduling the assembling of achannel's content; assembling the channel's content according to theschedule; compressing a subset of the URL data items, wherein each URLdata item is compressed individually independent of other URL data itemssuch that each compressed URL data item can be decompressed withoutdecompressing other URL data items; fragmenting the channel's contentinto packets; and multicasting the packets via a multicast network to aplurality of receivers.
 78. The method of claim 77, wherein the URL dataitems are compressed with a lossless data compression algorithm.
 79. Themethod of claim 77, wherein the step of assembling the channel's contentfurther comprises the steps of: assembling a base package of thechannel's content, wherein the base package contains each URL data itemin the channel; and assembling a delta package of the channel's content,wherein the delta package contains URL data items which have changed orare new since the previous assembling of the base package.
 80. Themethod of claim 79, wherein the step of scheduling the assembling of thechannel's content comprises scheduling the assembling of the basepackage and scheduling the assembling the delta package.
 81. The methodof claim 80, further comprising the step of difference compressing asubset of the URL data items in a channel's content which is present inboth the delta package and the previous base package
 82. The method ofclaim 81, wherein the step of difference compressing further comprisesthe steps of dividing a URL data item in the delta package intosections; and for each section, placing into a compressed version of theURL data item, one of a reference to where that section of content canbe found in the base package, or the section of the URL data item fromthe delta package.
 83. The method of claim 82, wherein the reference towhere the section of URL data item can be found in the base package isan offset from a beginning of the URL to a first byte and an offset to alast byte being referenced.
 84. The method of claim 79, furthercomprising the step of assembling a second delta package which containsURL data item which has changed since the assembling of the previousdelta package.
 85. The method of claim 77, further comprising the stepof encrypting a subset of a channel's packets prior to transmission,wherein either all or part of the packet are encrypted and wherein eachchannel's packets are encrypted with a set of encryption keys which areunique to that channel.
 86. A method for transmitting content organizedinto channels, wherein a channel's content includes a plurality of URLdata items and each URL data item is addressed by a URL, the systemcomprising. assembling a base package of a channel's content, whereinthe base package contains each URL data item in the channel, fragmentingthe base package into packets; multicasting the base package packets toa plurality of receivers, assembling a delta package of a channel'scontent, wherein the delta package contains URL data items which havechanged or are new since the previous assembling of the base package;fragmenting the delta package into packets, and multicasting the deltapackage packets to the plurality of receivers
 87. The method of claim86, further comprising the step of scheduling the assembling of basepackages and delta packages, wherein the base packages and deltapackages are assembled according to the schedule.
 88. The method ofclaim 86, further comprising the step of scheduling the multicasttransmission of base package packets and for scheduling subsequentperiodic multicast transmission of delta package packets, wherein thebase package packets and delta package packets are multicast accordingto the schedule 89 The method of claim 88, where in base package packetsare scheduled for transmission at a time when the receiver is not likelyto be in use for other applications.
 90. The method of claim 86, furthercomprising the step of compressing a subset of the URL data items in thebase and delta packages, wherein each URL data item is compressedindividually independent of other URL data items such that eachcompressed URL data item can be decompressed without decompressing otherURL data items.
 91. The method of claim 90, wherein the URL data itemsare compressed with a lossless data compression algorithm
 92. The methodof claim 90, further comprising the step of difference compressing asubset of the URL data items which are present in both in the deltapackage and the previous base package.
 93. The method of claim 92,wherein the step of difference compressing further comprises: dividing aURL data item in the delta package into sections; and for each section,placing into a compressed version of the URL data item, one of areference to where that section can be found in the base package, or thesection of the URL data item from the delta package. 94 The method ofclaim 93, wherein the reference to where the section of URL data itemcan be found in the base package is an offset from a beginning of theURL to a first byte and an offset to a last byte being referenced. 95.The method of claim 93, further comprising compressing a subset of thepreviously difference compressed URL data items with a lossless datacompression algorithm 96 The method of claim 86, further comprising thestep of assembling a second delta package that contains URL data itemsthat have changed since the assembling of the previous delta package 97.A method for transmitting content organized into channels, wherein achannel's content includes a plurality of URL data items and each URLdata item is addressed by a URL, the method comprising the steps of:scheduling the assembling of a channel's content; assembling thechannel's content according to the schedule; fragmenting the channel'scontent into packets, multicasting the packets to a plurality ofreceivers, wherein each receiver stores the received channel's contentin a receiver memory; and receiving usage reports from each receiver,wherein each usage report identifies a subset of URL data items from thestored URL data items that was accessed from the receiver memory. 98.The method of claim 97, further comprising the step of organizing thereceived usage reports by channel.
 99. The method of claim 97, whereineach usage report contains information identifying a subset of URL dataitems delivered to a web browser.
 100. The method of claim 97, whereinthe usage reports comprise a set of files, and wherein the URL data itemaccessed for each channel is referenced in one set of files 101 Themethod of claim 97, wherein the usage reports contain informationidentifying each URL data item, from the stored URL data items, beingdelivered to a web browser.
 102. The method of claim 100, furthercomprising the step of performing usage reporting on a subset of achannel's URL data items and wherein the files contain a separate recordfor each time a usage reported URL data item was delivered to the webbrowser, and wherein the record identifies the URL of the URL data item.103. The method of claim 102, wherein the record identifies when the URLdata item was delivered to the web browser.
 104. The method of claim102, wherein the record contains a field uniquely identifying the userthat accessed the URL data item.
 105. The method of claim 104, whereinthe field uniquely identifying the user does not specify the identity ofthe user.
 106. The method of claim 104, wherein the field uniquelyidentifying the user specifies the identity of the user.
 107. The methodof claim 97, wherein a channel's content is assembled from a web serverand further comprising the step of notifying the web server from which aURL data item was assembled that the URL data item was accessed by auser.
 108. The method of claim 107, wherein the web server is notifiedthat the URL data item was accessed by a user by notifying the webserver that the URL data item was delivered to a browser.
 109. Themethod of claim 107, wherein the web server is notified that the URLdata item was accessed by initiating an HTTP GET operation for the URLdata item. 110 The method of claim 107, wherein the web server isnotified of multiple accesses of multiple URL data items by initiatingan HTTP PUT operation.
 111. The method of claim 107, wherein the webserver is notified of multiple accesses of multiple URL data items byinitiating an HTTP POST operation.
 112. The method of claim 107, whereinthe web server is notified that the URL data item was accessed bye-mail, and wherein multiple accesses of multiple URL data item isreported in one e-mail.
 113. The method of claim 97, further comprisingthe step of compressing a subset of the URL data items, wherein each URLdata item is compressed individually independent of other URL data itemssuch that each compressed URL data item can be decompressed withoutdecompressing other URL data items.
 114. A receiver for receiving from amulticast network content organized into channels, wherein a channel'scontent includes a plurality of URL data items and each URL data item isaddressed by a URL, and wherein the multicast network transmits thechannel's content to the receiver in packets, the receiver comprisingmeans for determining a multicast address used to carry a channel'spackets; means for enabling reception of packets containing a channel'smulticast address; means for receiving the packets containing achannel's multicast address; means for assembling the received packetsinto a channel's content; means for storing the channel's content; andmeans for allowing a user to access the stored channel's content. 115.The receiver of claim 114, wherein some of the received packets arewholly or partially encrypted and the receiver further comprises meansfor decrypting the encrypted packets using a decrypting key unique tothe channel.
 116. The receiver of claim 114, wherein the receiver isonly authorized to receive selected packets
 117. The receiver of claim114, wherein the channel's content is stored in a single file.
 118. Thereceiver of claim l 14, wherein the channel's content is stored in anumber of files, and wherein the number of files is less than the totalnumber of URL data items in the channel
 119. The receiver of claim 114,further comprising means for allowing the user to designate the channelsto be received.
 120. The receiver of claim 119, further comprising meansfor only receiving the user designated channels
 121. The receiver ofclaim 120, further comprising means for displaying to the user the setof channels which can be received
 122. The receiver of claim 121,further comprising means for receiving an electronic program guidechannel, wherein the content of the electronic program guide channelincludes channel selection information allowing the user to evaluatewhich channels the receiver should receive.
 123. The receiver of claim122, further comprising means for receiving updates for the electronicprogram guide channel.
 124. The receiver of claim 122, wherein thechannel selection information in the electronic program guide channelincludes a schedule for when the content of the channels will betransmitted.
 125. The receiver of claim 122, wherein the channelselection information in the electronic program guide channel includesan amount of memory space needed to store the channel's content
 126. Thereceiver of claim 114, further comprising means for determining whetherall the packets for a channel have been received.
 127. The receiver ofclaim 126, wherein the multicast network transmits packets to thereceiver more than once and further comprising means for determiningwhich packets for a channel were not received and assembling thechannel's missing packets from the retransmitted packets. 128 Thereceiver of claim 114, wherein the receiver comprises a personalcomputer.
 129. The receiver of claim 114, wherein the receiver comprisesa set top box
 130. The receiver of claim 1 14, wherein the receiver isintegrated with a digital television. 131 The receiver of claim 114,further comprising: means for determining when a URL data item requestedto be accessed by the user is not present within the stored channelcontent, means for notifying the user that the requested URL data itemis not present within the stored channel content, and means for allowingthe user to access the non-present URL data item via a connection to aTCP/IP network.
 132. The receiver of claim 131, wherein the TCP/IPnetwork comprises the Internet. 133 The receiver of claim 131, furthercomprising means for soliciting the user whether to access thenon-present URL data item via the connection to the TCP/IP network. 134.The receiver of claim 132, wherein the multicast network is ageosynchronous satellite broadcast system and wherein the connection tothe Internet is a dial-up modem.
 135. The receiver of claim 114, furthercomprising means for tracking each time the user accesses URL data itemsin the stored channel content. 136 The receiver of claim 135, furthercomprising means for reporting the tracked user accesses to a web sitefrom which the accessed URL data items were assembled.
 137. The receiverof claim 114, wherein the packet receiving means monitors receiveractivity and selectively receives packets based on the monitoredactivity.
 138. The receiver of claim 114, further comprising means forsoliciting the user to determine when packets should be received andwherein the packet receiving means selectively receives packets based onuser preferences 139 A receiver for receiving from a multicast networkcontent organized into channels, wherein a channel's content includes aplurality of URL data items and each URL data item is addressed by aURL, and wherein the multicast network transmits the channel's contentto the receiver in packets, the receiver comprising: means fordetermining a multicast address used to carry a channel's packets; meansfor enabling reception of packets containing a channel's multicastaddress; means for receiving the packets containing a channel'smulticast address; means for assembling the received packets into achannel's content; means for storing the channel's content; means forallowing a user to access the stored channel's content; and means forindividually decompressing each compressed URL data item in the storedchannel content at a time when the user accesses the URL data item. 140.The receiver of claim 139, wherein the URL data item is decompressed afirst time the user access the URL data item and further comprisingmeans for storing the decompressed URL data item
 141. The receiver ofclaim 139, wherein the URL data item is decompressed each time a useraccess the URL data item. 142 The receiver of claim 139, wherein themulticast network transmits a channel's content in base package packetsand delta package packets, and the means for assembling the base packagepackets into a complete base package and assembling the delta packagepackets into a complete delta package.
 143. The receiver of claim 142,wherein the means for storing the channel's content stores the completebase package for the channel and the complete delta package for thechannel.
 144. The receiver of claim 142, wherein the means for allowinga user to access the stored channel's content provides the user with aURL data item from a delta package when the URL data item is present ina delta package and provides the user a URL data item from a basepackage when the URL data item is not present in a delta package 145 Areceiver in a multicast system, comprising: means for receiving URL dataitems from a multicast network; means for storing the received URL dataitems; means for allowing a user to access the stored URL data items;and means for tracking user access to the stored URL data items
 146. Thereceiver of claim 145, wherein the URL data items are assembled from aweb site and further comprising means for reporting the tracked useraccess to the web site
 147. The receiver of claim 145, wherein thetracking means includes means for counting a number of times the useraccesses a subset of the stored URL data items.
 148. The receiver ofclaim 145, further comprising means for determining when a URL data itemrequested to be accessed by the user is not present within the storedURL data items, means for notifying the user that the requested URL dataitem is not present within the stored URL data items, and means forallowing the user to access the non-present URL data item via aconnection to a TCP/IP network.
 149. The receiver of claim 148, furthercomprising means for soliciting the user whether to access thenon-present URL data item via the connection to the TCP/IP network. 150.The receiver of claim 148, wherein the multicast network is ageosynchronous satellite broadcast system and wherein the connection tothe TCP/IP network is a dial-up modem.
 151. A receiver in a multicastsystem, comprising. means for monitoring receiver activity; and meansfor selectively receiving content from a multicast network, wherein thecontent is selectively received based on the monitored receiver activity152. The receiver of claim 151, wherein the monitoring means monitorswhether any other applications are currently active on the receiver 153.The receiver of claim 151, wherein the monitoring means monitorsutilization of a receiver memory
 154. The receiver of claim 151, whereinthe monitoring means monitors user activity on an input device of thereceiver.
 155. The receiver of claim 154, wherein the receiver is apersonal computer and the user activity comprises keystrokes on akeyboard input device
 156. The receiver of claim 154, wherein thereceiver is a personal computer and the user activity comprises clickson a mouse input device.
 157. The receiver of claim 156, wherein theuser activity further comprises keystrokes on a keyboard input device158. The receiver of claim 151, further comprising means for solicitinga user to specify when content should be received and wherein thereceiving means receives content based on the user specifications 159.The receiver of claim 159, wherein the user specifications include timeof day when content should be received.
 160. The receiver of claim 158,wherein the content comprises base packages and delta packages and theuser specifications includes a first time period when base packages canbe received and a second time period when delta packages can bereceived.
 161. The receiver of claim 151, further comprising means forsuspending reception of content when the monitoring means determinesthat reception will interfere with other receiver activity
 162. Thereceiver of claim 161, further comprising means for automaticallyenabling reception of content after the monitoring means determines thatreception will not interfere with other receiver activity.
 163. Thereceiver of claim 161, further comprising means for automaticallyenabling reception at a time of day when reception will most likely notinterfere with other receiver activity. 164 The receiver of claim 161,wherein the monitoring means determines that reception will notinterfere with other activity by monitoring user activity on an inputdevice of the receiver.
 165. The receiver of claim 164, wherein thereceiver is a personal computer and the user activity comprises clickson a mouse input device
 166. A receiver in a multicast system,comprising a package receiver for receiving packets containing URL dataitems from a multicast network and assembling the received packets intoa channel, wherein the channel comprises a set of URL data items; amemory for storing the channel, and a content viewer for allowing a userto request access to and access the URL data items in the stored channel167. The receiver of claim 166, further comprising a browser forsearching the memory for URL data items requested by the user.
 168. Thereceiver of claim 166, wherein the receiver comprises a personalcomputer.
 169. The receiver of claim 166, wherein the receiver comprisesa set top box.
 170. The receiver of claim 166, wherein the receiver isintegrated with a digital television.
 171. The receiver of claim 166,wherein the packets received from the multicast network are encryptedand the package receiver decrypts the packets.
 172. A system formulticasting URL data items from web sites to a plurality of receivers,comprising: a web crawler for retrieving URL data items from the websites and formatting the retrieved URL data items into packages; apackage delivery subsystem for receiving the packages from the webcrawler, fragmenting the packages into packets and transmitting thepackets to a multicast network; and a conditional access system fordetermining which receivers are authorized to receive the packets,wherein the multicast network multicasts the packets only to authorizedreceivers.
 173. The system of claim 172, wherein the web crawlerretrieves URL data items from the web sites according to a predeterminedchannel definition.
 174. The system of claim 172, wherein the multicastnetwork multicasts an electronic program guide to each receiver, andwherein the electronic program guide contains descriptions of the websites from which URL data items were retrieved. 175 The system of claim174, wherein a receiver uses the electronic program guide to subscribeto selected web sites and the system further comprises a registrationserver for tracking subscription information.
 176. The system of claim175, wherein the registration server provides the subscriptioninformation to the package delivery subsystem.
 177. The system of claim172, further comprising a cache hit tracker which receives usage reportsfrom the receivers, wherein the usage reports contain informationidentifying which URL data items, from the set of URL data itemsreceived by the receiver, were accessed by a user.
 178. The system ofclaim 177, wherein the cache hit tracker stores the usage reports in hitlog files.
 179. The system of claim 178, wherein the cache hit trackerprovides the hit log files to the web sites from which the URL dataitems were retrieved.
 180. The system of claim 172, wherein themulticast network multicasts the packets to the receiver over a one-wayhigh speed link.
 181. The system of claim 180, wherein the high speedlink comprises a satellite link.
 182. A system for multicasting contentorganized into channels to a plurality of receivers, wherein a channel'scontent includes a plurality of URL data items from at least one website, comprising: a web crawler for retrieving the URL data items fromthe web site via a TCP/IP network and formatting the retrieved URL dataitems into packages; a package delivery subsystem for receiving thepackages from the web crawler and fragmenting the packages into packets;a conditional access system for determining which receivers areauthorized to receive the packets; and a multicast network for receivingthe packets from the package delivery subsystem, wherein the conditionalaccess system encrypts the packets and the multicast network multicaststhe encrypted packets to the authorized receivers, and wherein theauthorized receivers store the packets in a memory and decrypt thepackets.
 183. The system of claim 182, wherein the web crawlercompresses a subset of the retrieved URL data items, and wherein eachURL data item is compressed individually independent of other URLcontent such that the receiver can decompress each URL data item withoutdecompressing other URL data items
 184. The system of claim 182, whereinthe web crawler assembles a base package containing each URL data itemin the channel and subsequently assembles a delta package containing URLdata items which have changed or are new since the previous assemblingof the base package.
 185. The system of claim 184, wherein the webcrawler assembles the base packages and delta packages according to aschedule. 186 The system of claim 184, wherein the multicast networkmulticasts the base packages and the delta packages according to aschedule.
 187. The system of claim 186, wherein the base packages arescheduled for multicasting at a time when the receiver is not likely tobe in use for other applications
 188. The system of claim 184, whereinthe web crawler compresses a subset of the retrieved URL data items, andwherein each URL data item is compressed individually independent ofother URL content such that the receiver can decompress each URL dataitem without decompressing other URL data items.
 189. The system ofclaim 188, wherein the web crawler difference compresses a subset of theURL data items that are present in both the delta package and theprevious base package
 190. The system of claim 189, wherein the webcrawler performs difference compression by: dividing a URL data item inthe delta package into sections; and for each section, places into acompressed version of the URL data item, one of a reference to wherethat section can be found in the base package, or the section of the URLdata item from the delta package
 191. The system of claim 184, whereinthe web crawler assembles a second delta package which contains URL dataitems which have changed since the assembling of the previous deltapackage.
 192. The system of claim 182, wherein each receiver comprises acontent viewer for allowing a user to access the stored URL data items.193. The system of claim 192, further comprising a cache hit trackerwhich receives usage reports from the receivers, wherein the usagereports contain information identifying which URL data items, from theset of stored URL data items, was accessed by a user.
 194. The system ofclaim 193, wherein the cache hit tracker provides the usage reports tothe web sites from which the accessed URL data items were retrieved.195. The system of claim 182, wherein the TCP/IP network comprises theInternet.
 196. The system of claim 182, wherein the multicast networkmulticasts the packets to the receiver over a one-way high speed link.